# Presentation: 136100

## Program/Project ManagementGovernment Specific Course II 

- 	Government Specific Course II

- 5-

- Program/Project ManagementGovernment Specific Course II

## Module 5 –
Technical Management Process

- 	Government Specific Course II

- 5-

- Module 5 –
- Technical Management Process

- 	Government Specific Course II

- 5-

- Topic 1-
- Introduction/Objectives

## Technical Management Process

- 	Government Specific Course II

- 5-

- Separated into two categories:

- Planning, Control and Assessment Processes
- Plans overall Systems Engineering effort
- Controls requirements, interfaces, risks, configurations and technical data
- Assesses, monitors and reviews planned progress
- Decision Analysis Process

## Managing Technical Processes

- 	Government Specific Course II

- 5-

- Technical Management Processes are what make Technical Processes successful.

- Technical Processes act to design and implement design of a product or system

## Decision Analysis Process

- 	Government Specific Course II

- 5-

- The basis for evaluating and selecting technical alternatives when decision needs to be made

- Requirements Development Process - Balance requirements in terms of cost, schedule, performance and risk
- Design Solution Process - Conduct analysis to help choose alternatives
- Verification/Validation Process – Determine cause of deficiencies and determining corrective actions

## Learning Objectives

- 	Government Specific Course II

- 5-

- At the end of this module, you will be able to:

- Describe the purpose, inputs and outputs, and activities of the Decision Analysis Process.
- Describe the different types of decision-making approaches

## Learning Objectives (cont.)

- 	Government Specific Course II

- 5-

- Describe the typical evaluation techniques.
- Describe the role of Modeling and Simulation (M&S) in Decision Analysis.

## Question

- 	Government Specific Course II

- 5-

- Why is the Decision Analysis Process so important when developing a system or product?

## Importance of the Process

- 	Government Specific Course II

- 5-

- Cross cutting Technical Management Process that provides a rational, repeatable basis for evaluating/selecting alternatives when decisions must be made

- When formal decision is needed or effectiveness of alternative must be determined
- Decisions made within dimensions of a trade space

- 	Government Specific Course II

- 5-

- Topic 2 –
- Approaches to Decision Making

## Formal Decision-Making

- 	Government Specific Course II

- 5-

- Done with structured process which is identified by a formal agreement or is specified in a guideline
- Results assessed at Technical Review
- Usually have major cost, schedule, risk or performance impacts
- Not used for non-critical decisions

## Informal Decision-Making

- 	Government Specific Course II

- 5-

- Follows same structured approach as formal decision making with less documentation and fewer rigors
- Topic being studied is usually of lesser importance and while not considered critical, may still be important to the technical effort

## Judgmental Decision-Making

- 	Government Specific Course II

- 5-

- Involves selection of alternative
- Uses less rigorous analysis
- Used for non-critical decisions and for smaller projects
- Used for the majority of technical decision-making opportunities

## “Structured” Decision-Making

- 	Government Specific Course II

- 5-

- “Structured” Decision-Making

- Essential that decision recommendations made and techniques, assumptions and considered impacts are captured and maintained
- Formal and informal decision-making are both considered structured approaches
- Key outputs of the Decision Analysis Process

## Decision Analysis Process Purpose

- 	Government Specific Course II

- 5-

- Evaluate as well as formulate alternatives
- Select the methods to be used in conducting the analysis
- Select criteria for deciding among them
- Totally risk-free decision are impossible
- Decisions are typically made under various levels of uncertainty.
- Identified risks influencing decisions must be tracked and mitigated

## Objective – Decision Analysis

- 	Government Specific Course II

- 5-

- To conduct appropriate assessments and evaluations to assure that informed decisions can be made from a set of alternatives

- Must be made within the constraints of established product performance requirements and cost, schedule and risk criteria

## Audit Trail of Decisions

- 	Government Specific Course II

- 5-

- Include in the recording and maintaining:
- Decision approach used
- Assumptions
- Recommendations
- Impact of each recommendation
- Rationale for selection criteria used
- Selected alternatives

## Factors for Decision Making

- 	Government Specific Course II

- 5-

- Budget
- User requirements
- Schedules
- Risks
- Trade options
- Constraints

## Importance of Decision-Making

- 	Government Specific Course II

- 5-

- Decisions made regarding technical activities are particularly important
- Must be made in a rational way using a repeatable, structured process with collateral assumptions and impacts fully understood

## Structured Method

- 	Government Specific Course II

- 5-

- There is a structured method used to analyze possible decision alternatives

- Used to evaluate properly identified alternatives against a set of established, stable criteria
- Establish guidelines as to when a method should be used are essential

## Structured Approach 

- 	Government Specific Course II

- 5-

- Structured Approach

- Advantages include:
- Reducing the subjective nature of decision
- Improving probability of selecting solution that meets multiple expectations
- Helping to reduce uncertainty by having reasoned alternatives
- Permit re-assessment if criteria, assumptions, requirements or expectations change

## Inputs

- 	Government Specific Course II

- 5-

- Alternatives provide three necessary inputs to decision-makers.  These include:

- Set of recommended actions that could be taken to help solve issue or problem
- Impact of each recommendation
- Documentation that presents evaluation methods and criteria used, alternatives considered, assumptions made, and any decision rational

## Decision Analysis Role

- 	Government Specific Course II

- 5-

- Plays an important role in both Systems Engineering and Technical Processes

- Also supports the various other Technical Management  Processes used to control Systems Engineering activities

## Question

- 	Government Specific Course II

- 5-

- What is the advantage of using a structured Decision Analysis Process?

## Decision Analysis Activities

- 	Government Specific Course II

- 5-

- Used to prepare for Decision Analysis Implementation
- Creates the Decision Needs and Issues/Problems
- Evaluation Criteria established; Analysis Approach, Criteria, and Techniques chosen

## Activities (cont.)

- 	Government Specific Course II

- 5-

- Evaluation Technique is chosen
- Support Data needed to conduct evaluation of alternatives
- Evaluation of Alternatives provides the Analysis Results

## Step 1

- 	Government Specific Course II

- 5-

- The first step is Preparing for Decision Analysis Implementation.

- Establish guidelines
- Prepare a decision analysis strategy for technical decision-making
- Obtain strategy, plans or guideline commitment
- Prepare Procedures for implementation

## Step 1 (cont.)

- 	Government Specific Course II

- 5-

- Establish/acquire automated tools that will help with implementation of methods
- Train IPTs and other technical personnel in decision analysis method
- Outcomes include:
- Guidelines for when to apply formal evaluation process
- Procedures for performing decision analysis process
- Availability of qualified personnel

## When to use Structured Evaluation

- 	Government Specific Course II

- 5-

- When the decision:
- Involves moderate to high risk alternatives
- Is related to changing work products under configuration management control
- Would cause schedule delays that exceeded allowable set limits
- Affects the ability to achieve technical objectives
- Analysis costs are reasonable when compared to the decision's impact
- Will address a technical issue that could impact the program's Acquisition Program Baseline (APB)

- 	Government Specific Course II

- 5-

- Topic 3 –
- Decision Analysis Process

## Prepare for Implementation

- 	Government Specific Course II

- 5-

- Prepare for Decision Analysis Implementation establishes framework for Decision Analysis Process

- Overview of activities in this process:
- Establish an analysis/evaluation approach including:
- Defining the problem/issue for which a decision is needed
- Defining analysis goals, ground rules, constraints
- Identifying feasible alternatives
- Defining an approach to resolve the problem/issue

## Overview of Activities

- 	Government Specific Course II

- 5-

- Establish Evaluation Criteria
- Defining the appropriate/applicable measures of effectiveness
- Defining the variables that must be considered
- Identifying data needs, existing data, expected values, sources of data, etc.
- Identifying uncertainties and levels of risks

## Overview of Activities (cont.)

- 	Government Specific Course II

- 5-

- Establish the evaluation technique
- Select from known techniques
- Define modeling requirements
- Collect needed supporting data
- Utilize existing or input data accompanying the issue/problem
- Generate new data through analysis, as needed

## Overview of Activities (cont.)

- 	Government Specific Course II

- 5-

- Conduct evaluation of alternatives
- Perform steps in the model or run the model
- Perform sensitivity analysis
- Report decision analysis results
- Recommendation(s) and impacts
- Levels of confidence in the validity of the results
- Sensitivities to assumptions, uncertainties and risks
- Trade considerations

## Question

- 	Government Specific Course II

- 5-

- Which of the following situations probably indicate that the use of a formal Decision Analysis Process is warranted?  Choose all that apply.

- Involves known low risk alternatives
- Potential impacts to the Acquisition Program Baseline
- Involves technical issues that will significantly impact development schedule
- Affects the ability to achieve key technical objectives

## Establish Analysis Approach

- 	Government Specific Course II

- 5-

- This first step involves:
- Defining decision need, issue or problem
- Defining analysis goals, ground rules, and constraints
- Identifying feasible options
- Perform a literature search
- Solicit alternatives from relevant stakeholders
- Use brainstorming sessions

## Establish Analysis Approach (cont.)

- 	Government Specific Course II

- 5-

- Define the approach to the decision solution considering constraints
- Document the proposed analysis approach
- Expected Outcomes:
- Defined analysis approach
- Set of alternatives listed

## Establish Evaluation Criteria

- 	Government Specific Course II

- 5-

- Tasks for this step include:
- Defining the minimum needed evaluation criteria for evaluating alternatives
- Measures relationship to decision need
- Technology/limitations, impacts, risks, affordability
- Interoperability, open systems adaptability, robustness
- Defining scale or priority weighting scheme for ranking relative importance of evaluation criteria

## Establish Evaluation Criteria (cont.)

- 	Government Specific Course II

- 5-

- Ranking or ordering criteria according to the defined scale
- Assessing criteria as to whether it is a constraint or variable that can be used in a trade-off analysis to determine relative importance
- Review and modify the criteria as appropriate
- Identify data needs, sources of data and how to obtain
- Identify uncertainty and risks

## Establishing Criteria (cont.)

- 	Government Specific Course II

- 5-

- Document rationale for selection and rejection of evaluation criteria
- Expected Outcomes:
- Set of valid evaluation criteria
- Ranking of criteria importance

## Question

- 	Government Specific Course II

- 5-

- What decision analysis approach is applicable to the scenario?

## Evaluation Technique

- 	Government Specific Course II

- 5-

- Tasks in this step include:
- Selecting an appropriate technique or method for performing the decision analysis, based on:
- Cost, time, and risk
- Purpose of analysis
- Availability of needed data
- Ability of technique to focus
- Ability of technique to consider criteria defined

## Evaluation Technique (cont.)

- 	Government Specific Course II

- 5-

- Determining which measures/data are needed
- Determining whether a model is needed
- If model is needed and does not exist, generate one
- Tailor model to specific decision analysis evaluation need

## Evaluation Technique (cont.)

- 	Government Specific Course II

- 5-

- Documenting techniques/methods selected and provide rationale for selection
- Expected Outcomes:
- One or more techniques/methods for conducting evaluation
- Rationale behind technique/method
- Model for executing technique
- Collection scheme for measures/data needed

## Question

- 	Government Specific Course II

- 5-

- What are some evaluation criteria applicable to the scenario?

## Methods/Techniques

- 	Government Specific Course II

- 5-

- Modeling and Simulation (M&S)
- Decision Tree Analysis
- Trade Studies

## Modeling and Simulation

- 	Government Specific Course II

- 5-

- DEFINITION:  The use of models, including emulators, prototypes, simulators, and stimulators, either statically or over time, to develop data as basis for making managerial or technical decisions

- Model - Physical, mathematical or logical representative of  system, entity, phenomenon or process
- Simulation – method for implementing a model over time

## Decision Tree Analysis

- 	Government Specific Course II

- 5-

- Explores alternative decision in a branched tree-like form

- Each alternative is looked at as branching from single point
- Then, each alternative has its own branches, and so on...
- Pros and cons or expected values can be used to evaluate each branch
- Will result in either a qualitative or quantitative result from which choice can be made

## Trade Studies

- 	Government Specific Course II

- 5-

- Provide a structured way to analyze and evaluate among alternatives
- Include:
- Operational Capabilities
- Functional and performance requirements
- Design alternatives and their related manufacturing, testing, and support processes
- Program schedule
- Life cycle costs

## Modeling/Simulation Usage

- 	Government Specific Course II

- 5-

- Virtual prototyping during early stages of development
- Assess interoperability with legacy, current, and future systems
- Examine reliability, availability, maintainability, transportability, provisioning, cost implications, and human-machine interface design considerations

## Modeling/Simulation Usage (cont.)

- 	Government Specific Course II

- 5-

- Support the Verification and Validation Technical Processes
- Help in designing manufacturing facilities, defining production flows to meet planned production rates, and eliminating production bottlenecks
- Examine support alternatives using logistical models

## Question

- 	Government Specific Course II

- 5-

- How can the modeling and simulation technique be applicable to the scenario?

## Question

- 	Government Specific Course II

- 5-

- How can the decision tree analysis technique be applicable to the scenario?

## Question

- 	Government Specific Course II

- 5-

- How can the trade studies technique be applicable to the scenario?

## Collecting Needed Support Data

- 	Government Specific Course II

- 5-

- Tasks in this step includes:
- Identifying the data needed to conduct the evaluation of alternatives
- Determining and utilizing existing data
- Generating new data through analyses
- Acquiring actual test data
- Validating that the data collected
- Collecting and recording identified, needed and available data

## Question

- 	Government Specific Course II

- 5-

- What supporting data is needed for the scenario?

## Question

- 	Government Specific Course II

- 5-

- What evaluation of alternatives techniques are applicable to the scenario?

## Evaluation of Alternatives

- 	Government Specific Course II

- 5-

- After valid data has been collected, conduct the evaluation of alternatives
- Assess the proposed alternative solutions using the previously-established evaluation techniques and criteria and within the applicable trade space
- Trade Space:  Set of program and system parameters, attributes and characteristics required to satisfy system performance

## Evaluation of Alternatives (cont.)

- 	Government Specific Course II

- 5-

- Evaluation may include:
- Analyzing each alternative for mission and cost-effectiveness
- Analyzing each alternative for Total Ownership Cost (TOC)
- Analyzing environmental and organizational impacts
- Determining if there are opportunities to reduce costs, improve performance or shorten schedule

## Tasks

- 	Government Specific Course II

- 5-

- Tasks include:
- Assessing the assumptions related to evaluation criteria to ensure validity
- Assessing the sensitivity/uncertainty in the values resulting from the evaluation
- Performing simulations, modeling, prototypes, and pilots as necessary to exercise the evaluation criteria, evaluation techniques/methods, and alternative solutions

## Tasks (cont.)

- 	Government Specific Course II

- 5-

- Considering the use of new solutions, criteria and techniques if the proposed alternatives do not evaluate well
- Documenting the results of the evaluation of alternatives
- Expected Outcomes
- Recording of evaluation results
- Establishing of the overall system or end product effectiveness
- Establishing of the impact on enabling product requirements with respect to each life cycle support strategy

## Trade Space

- 	Government Specific Course II

- 5-

- Set of program and system parameters, attributes and characteristics required
- Decision-makers define and refine the system by making trade-offs
- Recommended decisions must fall inside trade space to be valid

## Critical Effectiveness Analyses

- 	Government Specific Course II

- 5-

- Mission Effectiveness
- The right product?
- Capability
- Able to get the job done?
- Mission Suitability
- Is it operable and available to perform?
- Cost Effectiveness
- How valuable is it?

## Question

- 	Government Specific Course II

- 5-

- What decision reporting would be applicable to the scenario?

## Reporting Results

- 	Government Specific Course II

- 5-

- Final step is to report the Decision Analysis results
- Tasks include:
- Select the alterative solution that best satisfies criteria
- Assess and monitor risks
- Form recommendations to be made to include:
- Confidence levels
- Implementation impacts

## Reporting Results (cont.)

- 	Government Specific Course II

- 5-

- Documenting the recommendation and supporting data
- Expected Outcome:
- Documentation of the decision analysis results and the recommendations made to respond to the decision need/issue or problem

- 	Government Specific Course II

- 5-

- Topic 4 –
- Review

## Learning Objectives

- 	Government Specific Course II

- 5-

- Recall the learning objectives for this module:
- Describe the different types of decision-making approaches
- Describe the purpose, inputs and outputs, and activities of the Decision Analysis Process.
- Describe the typical evaluation techniques.
- Describe the role of Modeling and Simulation (M&S) in Decision Analysis.

## Key Points (cont.)

- 	Government Specific Course II

- 5-

- Technical Management Process made up of two processes:

- Planning, Control and Assessment Processes
- Decision Analysis Process
- (we focused on the latter)

## Key Points (cont.)

- 	Government Specific Course II

- 5-

- Not possible to alleviate all risk
- Essential to conduct sensitivity analyses on the evaluation results and on the assumptions made

## Key Points (cont.)

- 	Government Specific Course II

- 5-

**Decision Analysis Process**
**Inputs:**  Decision Analysis plan and policies; needs, issues and problems to addressed along with relevant supporting data; techniques available in which IPTs are trained to use; and appropriate models that could be employed
**Outputs:**  Strategy for Decision Analysis including a set of guidelines for when structured decision analyses are required to be used, and the plans and procedures for doing implementation; the analysis approach, criteria and techniques to be used; results of analysis including assumptions, sensitivity analyses, etc. done as part of the analysis; and recommendations with their impacts

## Key Points (cont.)

- 	Government Specific Course II

- 5-

- Steps are to:
- Prepare for Decision Analysis Implementation
- Establish the Analysis Approach
- Establish Evaluation Criteria
- Establish the Evaluation Technique
- Collect the needed supporting data
- Conduct the evaluation of alternatives
- Report the decision analysis results

## Key Points (cont.)

- 	Government Specific Course II

- 5-

**Trade Space**
- Decision-makers define and refine the system by making trade-offs or other decision-making techniques with regard to cost, schedule, risk and technical performance---all of which must fall within the 'trade space.'

## Key Points (cont.)

- 	Government Specific Course II

- 5-

- Typical Structured Evaluation Techniques
- Modeling and Simulation (M&S)
- Decision Tree Analysis
- Trade Study

## Key Points (cont.)

- 	Government Specific Course II

- 5-

- Effective decision analysis requires the following:

- Clearly defined description of issue or problem
- Structured evaluation methods and procedures to use in implementation
- Identified alternatives
- Established evaluation criteria

## Key Points (cont.)

- 	Government Specific Course II

- 5-

- Modeling and Simulation
- Significant cost savings and reductions in the development schedule can be gained
- During pre-concept studies a properly constructed model and simulation of that model can do a top-down design analysis and a bottom-up realization analysis, yielding system potential operational and cost-effectiveness evaluations
- Time and cost savings of this approach can enhance system development and provide better decision-making