# Presentation: 461049

## Exploration Systems RFI 04-01:Methodology and Results

***Exploration Systems RFI 04-01:******Methodology and Results***

## RFI Evaluation Process: Phase I Scoring

**Design Principles, Objectives, and Guidelines**

**Crosscutting Design Drivers/ Architecture Elements**

**Program Management, Acquisition, and Interfaces**

_***STEP 1:***_
***Contractors submit white papers in three categories (responses due NLT ***_***5/20)***_

**HR&T Evaluation Team 1**
**(~60 Heads)**

**HR&T Evaluation Team 2**
**(~60 Heads)**

**Constellation Evaluation Team**
**(~10 Heads)**

_***STEP 2: ***_
***Eval teams utilize web-based document review tool in OExS Product Data Management system to assign scores and WBS/tech tags as file attributes (5/20-31)***

_***STEP 3: ***_
***Scored RFI responses are passed to teams in Requirements, Human & Robotic Technology, and Constellation for use in program content and management definition***

**Demonstrated Effectiveness / Technical Maturity**
**1- 5**

**Innovativeness / Variation from Historical Approach**
**1- 5**

**Potential Improvement in: Schedule, Cost, Risk**
**1- 5**

**Scoring**
**Criteria**

**RFI Review Period: May 20 – 31 **

## RFI Evaluation Process: Phase I Focus Area Mapping

**Results are mapped into**
**RFI Focus Areas to facilitate**
**retrieval and use for**
**subsequent BAAs and RFPs**

## RFI Evaluation Process: Phase I WBS Mapping

***RFI Evaluation Process: Phase I WBS Mapping***

**Results are mapped**
**to WBS Tier 1 to facilitate**
**retrieval and use for**
**subsequent BAAs and RFPs**

## RFI Evaluation Process: Phase I Tech Tagging

***RFI Evaluation Process: Phase I Tech Tagging***

**RFI responses are tagged for relevance to certain technology types to facilitate retrieval and use for subsequent BAAs and RFPs**

## Web-Based Evaluation System Process Map

**Click on the any of the above process blocks or arrows to see details**
**At anytime you can get to your “Review Task” through your work-list**

***Web-Based Evaluation System Process Map***

- Email notification

- RFI Portal
- WWW Browser:
- https://naccsli3.nis.nasa.gov

- Work-list

- Review Task Page

- Original RFI submission page
-  Download document
-  Review document

- Review Completed Evaluation

- Perform Task Complete

_***STEP 1: ***_
_***Login to the RFI Portal***_

_***STEP 2:***_
_***Get your Review Task***_

_***Task a:***_

_***Task b:***_

**START**

**START**

**START**

**START**

- Create RFI Review Document
-  Fill out evaluation criteria

_***Task c:***_

_***Task d:***_

_***STEP 3:***_
_***Complete your Evaluation***_

**Login**

**Login**

**Home**

**FAQs**

**Map**

## Scoring Criteria:Demonstrated Effectiveness / Technological Maturity

_**SCORE OF “1”:**_** This paper identifies concepts, technologies, or approaches that *****comply with the basic principles of physics, systems integration, or program management, but that have not been fully formulated*****.**
_**SCORE OF “2”:**_** This paper identifies concepts, technologies, or approaches that *****have been formulated through analytical or experimental proof-of-concept*****, but that have not been validated in a relevant environment or small-scale project.**
_**SCORE OF “3”:**_** This paper identifies concepts, technologies, or approaches that *****have been validated in a relevant environment or small-scale project*****, but have not been demonstrated in full prototype form or in the development phase of a major program.**
_**SCORE OF “4”:**_** This paper identifies concepts, technologies, or approaches that *****have been demonstrated in full prototype form or in the development phase of a major program*****, but have not been carried through the full life cycle of a development program or mission operations.**
_**SCORE OF “5”:**_** This paper identifies concepts, technologies, or approaches that *****have been successfully carried through the full life cycle***** of a development program and/or mission operations.**

## Scoring Criteria:Innovativeness / Variation from Historical Approach

_**SCORE OF “1”:**_** This paper identifies concepts, technologies, or approaches that have been *****broadly employed in previous development programs, that met the baseline requirements of these programs*****, but did not substantially improve functional performance or state-of-the-art.**
_**SCORE OF “2”:**_** This paper identifies concepts, technologies, or approaches that have been broadly employed in previous development programs with the *****effect of substantially exceeding baseline requirements and/or substantially improving functional performance or state-of-the-art******.***
_**SCORE OF “3”:**_** This paper identifies concepts, technologies, or approaches that have been *****employed occasionally***** in previous development programs with the effect of substantially exceeding baseline requirements and/or substantially improving functional performance or state-of-the-art.  Though they proved to be useful assets to the programs that employed them, *****these concepts, technologies, or approaches have not been infused into a broad number or diversity of programs*****.**
_**SCORE OF “4”:**_** This paper identifies concepts, technologies, or approaches that have been *****previously proposed***** for infusion into development programs.  These concepts, technologies, or approaches would potentially alter functional performance or state-of-the-art to the extent that they *****would fundamentally alter previously conducted system-wide trades in development programs*****.  These concepts, technologies, or approaches may have advocates in the development community, but are not broadly known or accepted.  Alternatively, the paper may *****propose the utilization of concepts, technologies, or approaches that have not been previously considered for their potential contributions to the capabilities of space systems*****.**
_**SCORE OF “5”:**_** This paper identifies concepts, technologies, or approaches that *****have not been previously proposed*****.  These concepts, technologies, or approaches would potentially alter functional performance or state-of-the-art to the extent that they would fundamentally alter system-wide trades in development programs.  Alternatively, the paper may *****propose concepts, technologies, or approaches that have not been considered as potential elements of a space system or space exploration architecture, and that might revolutionize the capabilities of developed systems*****.**

## Scoring Criteria:Potential Improvement in: Cost, Schedule, Risk

_**SCORE OF “1”:**_** This paper identifies concepts, technologies, or approaches that have been employed in the past and that *****caused programs to fail to meet baseline targets for cost, schedule, and/or risk*****.  Alternatively, the concepts, technologies, or approaches advocated by this paper have *****not been previously employed but present unjustifiable threats to a program that would be run within cost, schedule, and/or risk constraints*****.**
_**SCORE OF “2”:**_** This paper identifies concepts, technologies, or approaches that have been employed in the past and that *****posed substantial threats to***** *****baseline targets for cost, schedule, and/or risk*****.  If not previously employed, the concepts, technologies, or approaches advocated by this paper would force programs to accept substantial threats to***** ******cost, schedule, and/or risk baselines*****.**
_**SCORE OF “3”:**_** This paper identifies concepts, technologies, or approaches that *****helped enable previous programs to meet baseline targets for cost, schedule, and/or risk*****.  If not previously employed, the concepts, technologies, or approaches advocated by this paper *****would have a neutral effect on the cost, schedule, and/or risk baselines of programs exhibiting conventional analogs of the concepts, technologies, or approaches*****.**
_** **__**SCORE OF “4”:**_** This paper identifies concepts, technologies, or approaches that *****allowed previous programs to positively exceed baseline targets for cost, schedule, and/or risk*****.  If not previously employed, the concepts, technologies, or approaches advocated by this paper have a high probability of *****substantially improving the cost, schedule, and/or risk performance of programs employing conventional analogs*****.**
_**SCORE OF “5”:**_** This paper identifies concepts, technologies, or approaches that *****would reduce the cost, schedule, and/or risk of developing a system so much that the total system-of-systems or proposed acquisition strategy would be fundamentally improved or accelerated******.***** **

## RFI Results: Summary Observations

- Wealth of great material will be available in database.
- Comprehensive trade studies or architectural concepts
- Articulations of difficult lessons learned in prior NASA programs
**Engineering data management process and IT a major focus.**
- Some of these included collaboration tools
- Others extended this to modeling and simulation
**A recurring theme was the strong recommendation to consider and demonstrate technology *****X***** early in the lifecycle.**
- Examples of *X* included EVA suits, autonomous robotics, vehicle health management, simulation infrastructure, software development tools, and many others
**It was difficult for many respondents to treat this as an RFI, not an RFP.  Many respondents submitted what amounted to proposals to commence or resume technical studies or development programs.**
**Many submissions were recycled technical reports. Though content did not respond directly to the RFI focus areas, many were relevant.**

## RFI Results: High Score Distribution

## RFI Results: Design Principles Distribution

## RFI Results: Cross-Cutting Drivers Distribution

## RFI Results: Program Management Distribution

## RFI Results: WBS Distribution

***RFI Results: WBS Distribution***

***Systems Engineering & Integration tagged for >50%***
***of Evaluated Submissions***

## Phase I Feedback to RFI Submitters 

_**Notification of Evaluation**_
- Online evaluation database will automatically notify submitter when RFI submission has been fully evaluated.

_**Results Analysis Posting**_
- Team of Headquarters managers will query database to: (1) identify exceptional proposals for concepts, technologies, or approaches; (2) review metadata that describes total complement of submissions; (3) probe certain clusters of papers based on mapping to WBS, focus areas, technology types, and scoring criteria.  A document describing the results of this analysis will be posted to the Exploration Systems Acquisition Information website by June 15.

_**Briefing of Results Analysis at Industry Day**_
- RFI Evaluation Process managers will brief results analysis at NASA Headquarters Industry Day, to be held on June 18.

## WBS Relevance Mapping – Phase II Utilization 

**RFI responses are passed to teams in Requirements, Human & Robotic Technology, and Constellation for use in program content formulation and management definition**

**H&RT BAA/NRA: System-of-Systems Challenges**

**H&RT BAA/NRA: Technology Gap Filling**

**Constellation Program Structure & Management, Acquisition Strategy, CEV RFP**

**Requirements Formulation and Architecture Definition Activities**

**BAA: Concept Exploration & Refinement**