**DOWNLOADED AND/OR HARD COPY UNCONTROLLED**

Verify that this is the correct version before use.

  ----------------------------- -------------------------- --------------
  **APPROVAL SIGNATURES**                                  **DATE**

  Gregory Blaney (original      IMS Representative         02/09/2009
  signature on file)                                       

                                                           
  ----------------------------- -------------------------- --------------

+---------+------------------------------+--------------+-------------+
| **R     |                              |              |             |
| EVISION |                              |              |             |
| HI      |                              |              |             |
| STORY** |                              |              |             |
+=========+==============================+==============+=============+
| R       | Description of Change        | Author       | Effective   |
| evision |                              |              | Date        |
+---------+------------------------------+--------------+-------------+
| Initial | Initial Release              | Cynthia      | 08/22/1997  |
|         |                              | Calhoun      |             |
|         |                              | IT/309       |             |
+---------+------------------------------+--------------+-------------+
| A       | Added 5.2 Process            | Cynthia      | 04/09/1998  |
|         | Verifications.               | Calhoun      |             |
|         |                              | IT/309       |             |
|         | Added NASA Policy Guideline  |              |             |
|         | NPG 1441.1 to References.    |              |             |
|         | Added section 8 (Metrics).   |              |             |
|         |                              |              |             |
|         | Format and content changes.  |              |             |
+---------+------------------------------+--------------+-------------+
| B       | Format changes               | Cynthia      | 05/05/1998  |
|         |                              | Calhoun      |             |
|         |                              | IT/309       |             |
+---------+------------------------------+--------------+-------------+
| C       | Added "Financial Management" | Cynthia      | 07/23/1998  |
|         | section.                     | Calhoun      |             |
|         |                              | IT/309       |             |
|         | Added references to          |              |             |
|         | 53.IT.0009-4 SLP in section  |              |             |
|         | 6.1.7.2.                     |              |             |
|         |                              |              |             |
|         | Modified sections 1.0,       |              |             |
|         | 6.1.2, 6.1.4.3, 6.1.7.1,     |              |             |
|         | 6.1.7.2, 6.1.7.3, 6.1.8, 6.2 |              |             |
+---------+------------------------------+--------------+-------------+
| D       | Quality Record - format      | Cynthia      | 08/26/1998  |
|         | change                       | Calhoun      |             |
|         |                              | IT/309       |             |
+---------+------------------------------+--------------+-------------+
| E       | Revised the Resource Manager | Cynthia      | 04/16/1999  |
|         | responsibility.              | Calhoun      |             |
|         |                              | IT/309       |             |
|         | Added the OSMA Annual Review |              |             |
|         | Report, OSMA Software        |              |             |
|         | Program Level 1 Technical    |              |             |
|         | Program Plan, and OSMA       |              |             |
|         | Software Program Operating   |              |             |
|         | Plan as Quality Records.     |              |             |
|         |                              |              |             |
|         | Updated Center Initiative    |              |             |
|         | Management Process to        |              |             |
|         | reflect the changes in the   |              |             |
|         | Level 1 Technical Program    |              |             |
|         | Plan.                        |              |             |
+---------+------------------------------+--------------+-------------+
| F       | References to Ames Quality   | Cynthia      | 09/10/1999  |
|         | Manual replaced with         | Calhoun      |             |
|         | references to IV&V Facility  | IT/309       |             |
|         | Quality Manual               |              |             |
+---------+------------------------------+--------------+-------------+
| G       | Reference to Process         | Cynthia      | 03/10/2000  |
|         | Verification Work            | Calhoun      |             |
|         | Instruction (IVV 09-3-1) in  | IT/309       |             |
|         | section 6.2                  |              |             |
+---------+------------------------------+--------------+-------------+
| H       | Format and Number changes;   | Griggs       | 12/06/2000  |
|         | Delete Reference to Ames     |              |             |
|         | Research Center              |              |             |
+---------+------------------------------+--------------+-------------+
| I       | Rewrite of document to       | Shandra      | 07/02/2001  |
|         | reflect refinement of        | Auvil        |             |
|         | procedure resulting from     |              |             |
|         | transfer to GSFC             |              |             |
+---------+------------------------------+--------------+-------------+
| J       | Changed item 6.5             | Wesley       | 05/15/2002  |
|         |                              | Deadrick     |             |
+---------+------------------------------+--------------+-------------+
| K       | Rewrite to reflect NRA and   | Frank        | 04/08/2003  |
|         | DDF scope additions          | Gmeindl      |             |
+---------+------------------------------+--------------+-------------+
| L       | Updated reference documents  | Wes Deadrick | 11/03/2004  |
|         | to include relevant          |              |             |
|         | standards. Evaluated and     |              |             |
|         | updated the metrics          |              |             |
|         | collected as a result of     |              |             |
|         | this process.                |              |             |
+---------+------------------------------+--------------+-------------+
| M       | Aligned with current         | Frank        | 06/08/2005  |
|         | practice                     | Gmeindl      |             |
+---------+------------------------------+--------------+-------------+
| N       | Aligned with other IV&V      | Lisa         | 02/07/2006  |
|         | Facility documentation       | Montgomery   |             |
+---------+------------------------------+--------------+-------------+
| O       | Changes from 08/11/2006      | Frank        | 08/16/2006  |
|         | audit                        | Gmeindl      |             |
+---------+------------------------------+--------------+-------------+
| P       | Remove references to         | Frank        | 09/07/2007  |
|         | cancelled 09-3-4, remove     | Gmeindl      |             |
|         | record, add research         |              |             |
|         | infusion initiatives, delete |              |             |
|         | RCT                          |              |             |
+---------+------------------------------+--------------+-------------+

  ---------- ------------------------------ --------------- --------------
  Q          Update process flow diagrams   Stephanie       08/05/2008
             to align with Facility         Ferguson        
             Management paradigm                            

  R          Changed "IV&V Facility" to     Stephanie       02/09/2009
             "IV&V Program"; updated to     Ferguson        
             align with current NASA IV&V                   
             roles and other documentation;                 
             updated Section 6.0, *Records*                 

                                                            
  ---------- ------------------------------ --------------- --------------

  ----------------------- -----------------------------------------------
  **REFERENCE DOCUMENTS** 

  Document                Title

  IVV QM                  IV&V Quality Manual

  IVV 06-2                Work Instruction for Processing Procurements
                          for Grants and Cooperative Agreements

  IVV 09-3-1              Work Instruction for Evaluating Research
                          Initiatives

  IVV 09-3-2              Work Instruction for Selecting OSMA SARP
                          Research Initiatives

  IVV 09-3-3              Work Instruction for Selecting IV&V Facility
                          Research Initiatives

  IVV 09-3-5              Work Instruction for Processing Research
                          Products

  IVV 09-3-6              Work Instruction for Conducting Research
                          Initiative Performance Reviews

  IVV 09-3-7              Work Instruction for the
                          Publication/Presentation of Research Results

  IVV 16                  Control of Records

  NPR 1441.1              NASA Records Retention Schedules

                          
  ----------------------- -----------------------------------------------

1.  **Purpose**

> The purpose of this system level procedure (SLP) is to establish
> guidance for defining and managing both the Office of Safety and
> Mission Assurance (OSMA) Software Assurance Research Program (SARP)
> and research undertaken for the NASA IV&V Program.

2.  **Scope**

> This SLP is applicable to:

1.  The day-to-day oversight and management of the SARP, sponsored by
    OSMA

2.  The NASA IV&V Program's own internally funded Research Program

> This SLP addresses research program planning; research proposal
> solicitation, evaluation and selection; and Research Initiative
> management, including the management and publication of results.

1.  **Research Funding**

> Funding for the OSMA SARP is managed by the NASA IV&V Program. Funds
> are used for Center Initiatives (CIs) and Research Infusion
> Initiatives (RIIs) conducted at various NASA locations. The funds'
> delivery cycle generally begins with funding allocation
> recommendations generated by the SARP. The recommendations are
> approved by OSMA, which in turn distributes the funding to the various
> NASA locations, which can include NASA IV&V. The NASA locations and
> NASA IV&V then distribute the funding to the corresponding
> initiatives.
>
> The NASA IV&V Program may choose to directly fund Facility Initiatives
> (FIs), which conduct research to improve IV&V methods and practices.
> These initiatives are often, though not exclusively, conducted at NASA
> IV&V.

**2.2 Types of Initiatives**

> The NASA IV&V Program manages, or has oversight of, four types of
> research projects:

1.  Center Initiatives (CI) -- OSMA SARP research conducted at NASA
    Centers or at NASA IV&V

2.  Facility Initiatives (FI) -- IV&V research funded by the NASA IV&V
    Program

3.  Research Infusion Initiative (RII) -- OSMA SARP projects conducted
    at NASA Centers or at NASA IV&V for the purpose of transferring
    technology that was developed under research into practice

    **2.3 Initiative Management Structure**

> The Chief of Research oversees all Research Initiatives. Each Research
> Initiative has a NASA Point of Contact (NPOC) and a Principal
> Investigator (PI).

The Chief of Research provides oversight to NPOCs and PIs. The NPOC
ensures that a PI's research products are accepted by NASA and are
retained in the Center Initiative Management (CIM) Tool. The PI actually
implements the research.

**2.3.1 NASA IV&V Management of the OSMA SARP**

> All NPOCs are civil service employees or NASA Jet Propulsion
> Laboratory (JPL) employees. NPOCs are located at NASA Centers or at
> JPL.
>
> Research Initiatives can be performed by NASA civil service employees,
> JPL employees, or contractors. If a Research Initiative is performed
> by a contractor, the NPOC is usually, although not exclusively, the
> Contracting Officer Technical Representative (COTR) for the
> contractual vehicle under which the work is conducted.
>
> The NASA Centers and JPL have their own contracting offices. The NPOCs
> for FIs and the Contracting Officer (CO) for FIs reside at NASA IV&V.
>
> PIs can be NASA civil service employees or contractors. PIs can be
> located at NASA Centers, JPL, NASA IV&V, or at contractor sites.

**2.3.2 Commonalities between OSMA SARP and IV&V-Funded Initiatives**

> The following are common features of the NASA IV&V Program\'s
> management of the OSMA SARP and NASA IV&V Program-funded research.

-   The CIM Tool is the data repository for all Research Initiative
    proposals, contracts, deliverables, and progress reports.

-   All Research Initiatives have a designated NPOC and a PI.

    **2.3.3 Differences Between OSMA SARP and IV&V-Funded Initiatives**

+---------------------+-------------------------+---------------------+
|                     | **SARP Initiatives**    | **DDF Initiatives** |
+---------------------+-------------------------+---------------------+
| **Selection         | Level 1 Plan -- Program | Notice of Intent    |
| Instrument**        | Operating Plan (POP)    | (NOI), Request for  |
|                     | call and/or NASA        | Proposal (RFP)      |
|                     | Research Announcement   |                     |
|                     | (NRA)                   |                     |
+---------------------+-------------------------+---------------------+
| **Selection         | Source Evaluation Board | Director and Chief  |
| Process**           |                         | of Research         |
+---------------------+-------------------------+---------------------+
| **Management        | OSMA delegates          | Director delegates  |
| Structure**         | day-to-day management   | day-to-day          |
|                     | to the NASA IV&V        | management to the   |
|                     | Program                 | Chief of Research   |
+---------------------+-------------------------+---------------------+
| **NPOC locations**  | NASA Centers            | NASA IV&V Program   |
|                     |                         | only                |
|                     | NASA IV&V Program       |                     |
+---------------------+-------------------------+---------------------+
| **Initiative        | CIs                     | FIs                 |
| types**             |                         |                     |
|                     | RIIs                    |                     |
+---------------------+-------------------------+---------------------+

**3.0 Definitions and Acronyms**

> Official NASA IV&V roles and terms are defined in the [Quality
> Manual](http://www.nasa.gov/centers/ivv/doc/170870main_IVV_QM.doc).
> Specialized definitions identified in this SLP are defined below.

3.  1.  **Acronyms**

+-----------+----------------------------------------------------------+
| CI        | Center Initiative                                        |
+-----------+----------------------------------------------------------+
| CIM       | Center Initiative Management                             |
+-----------+----------------------------------------------------------+
| FI        | Facility Initiative                                      |
+-----------+----------------------------------------------------------+
| FRC       | Federal Records Center                                   |
+-----------+----------------------------------------------------------+
| IMS       | NASA IV&V Management System                              |
+-----------+----------------------------------------------------------+
| JPL       | Jet Propulsion Laboratory                                |
+-----------+----------------------------------------------------------+
| NOI       | Notice of Intent                                         |
+-----------+----------------------------------------------------------+
| NPOC      | NASA Point of Contact                                    |
+-----------+----------------------------------------------------------+
| NPR       | NASA Procedural Requirement                              |
+-----------+----------------------------------------------------------+
| NRA       | NASA Research Announcement                               |
+-----------+----------------------------------------------------------+
| OSMA      | Office of Safety and Mission Assurance                   |
+-----------+----------------------------------------------------------+
| PI        | Principal Investigator                                   |
+-----------+----------------------------------------------------------+
| POP       | Program Operating Plan                                   |
+-----------+----------------------------------------------------------+
| QM        | Quality Manual                                           |
+-----------+----------------------------------------------------------+
| RFP       | Request for Proposal                                     |
|           |                                                          |
| RII       | Research Infusion Initiative                             |
+-----------+----------------------------------------------------------+
| RRWS      | Research Results Web Site                                |
+-----------+----------------------------------------------------------+
| SARP      | Software Assurance Research Program                      |
+-----------+----------------------------------------------------------+
| SAS       | Software Assurance Symposium                             |
+-----------+----------------------------------------------------------+
| SEB       | Source Evaluation Board                                  |
+-----------+----------------------------------------------------------+
| SLP       | System Level Procedure                                   |
+-----------+----------------------------------------------------------+
| SWG       | Software Working Group                                   |
+-----------+----------------------------------------------------------+

4.  **Process Flow Diagram**

> The following diagram depicts processes described in this document,
> and the responsibilities and actions that shall be performed by
> process participants. Any information supplemental to the depicted
> processes will appear after the diagram.
>
> ![](media/image1.wmf)

1.  **Determine Research Requirements**

> To determine research topics, the Chief of Research:

-   OSMA SARP topics

    -   Queries or discusses with NASA project personnel, NASA
        Headquarters personnel, Software Working Group (SWG) personnel,
        and Software Assurance Working Group personnel

-   NASA IV&V topics

    -   Queries stakeholders

> The Chief of Research documents and publishes selected research topics
> on the NASA IV&V web site in preparation for proposal solicitation.
>
> Please see IVV 09-3-1, *Work Instruction for Evaluating Research
> Initiatives*, for additional information.

1.  **Evaluate Ongoing Research**

> The Chief of Research conducts an annual evaluation of ongoing
> Research Initiatives to develop funding recommendations for the
> following year.
>
> Please see IVV 09-3-1, *Work Instruction for Evaluating Research
> Initiatives*, and IVV 09-3-6, *Work Instruction for Conducting
> Research Initiative Performance Reviews*, for additional information.

2.  **Select OSMA SARP Initiatives**

> The Chief of Research is responsible for the formulation and
> management of the SARP Evaluation Board (SEB). The SEB recommends
> funding for Research Initiatives (continued and new) based on proposal
> evaluations and amount of available funds. OSMA selects proposals for
> award based on the SEB recommendations.
>
> The Chief of Research drafts the Level 2 Plan, which defines the
> funding levels for ongoing Research Initiatives and serves as OSMA's
> selection of proposals for award.
>
> Please see IVV 09-3-2, *Work Instruction for Selecting OSMA SARP
> Research Initiatives*, for additional information.

3.  **Select Facility Initiatives**

> The Chief of Research solicits proposals when a need for research is
> identified by the NASA IV&V Program.
>
> The Chief of Research evaluates the proposals and prepares a
> recommendation for the NASA IV&V Director. The NASA IV&V Director
> approves the plan, thus selecting for award the proposals that offer
> the most benefit directly to the NASA IV&V Program.
>
> Please see IVV 09-3-3, *Work Instruction for Selecting IV&V Facility
> Research Initiatives*, for additional information.

4.  **Fund Ongoing Initiatives**

> NASA Centers fund existing grants or contracts according to the
> guidance contained in the approved Level 2 Plan. The assigned
> procurement officer funds existing contracts or grants and awards new
> grants or contracts for NASA IV&V.
>
> Please see IVV 09-3-1, *Work Instruction for Evaluating Research
> Initiatives*, or IVV 06-2, *Work Instruction for Processing
> Procurements for Grants and Cooperative Agreements,* for additional
> information.

5.  **Manage Research Initiatives**

> The Chief of Research, PIs, and NPOCs use the CIM Tool to manage
> deliverables, status reports, and performance reviews.
>
> Please see IVV 09-3-1, *Work Instruction for Evaluating Research
> Initiatives*; IVV 09-3-5, *Work Instruction for Processing Research
> Products;* and IVV 09-3-6, *Work Instruction for Conducting Research
> Initiative Performance Reviews*, for additional information.

6.  **Publish Results**

> Research results are published at the annual Software Assurance
> Symposium (SAS) and on the NASA IV&V Research Results Web Site (RRWS).
> The Chief of Research publishes deliverables that will provide
> significant value to the software assurance and engineering community.
>
> Please see IVV 09-3-7, *Work Instruction for the
> Publication/Presentation of Research Results*, for additional
> information.

4.  **Metrics**

> Any metrics associated with this SLP are established and tracked
> within the NASA IV&V Metrics Program.

5.  **Records**

> The following records will be generated and filed in accordance with
> this SLP and IVV 16, *Control of Records*, and in reference to NASA
> Procedural Requirement (NPR) 1441.1, *NASA Records Retention
> Schedules*.

+---------------+------+---+---------+---------------------+---------+
| **Record      | **Or | * | **Resp  | **Retention         | **Loc   |
| Name**        | igin | * | onsible | Requirement**       | ation** |
|               | al** | V | P       |                     |         |
|               |      | i | erson** |                     |         |
|               |      | t |         |                     |         |
|               |      | a |         |                     |         |
|               |      | l |         |                     |         |
|               |      | * |         |                     |         |
|               |      | * |         |                     |         |
+===============+======+===+=========+=====================+=========+
| OSMA SARP     | Y    | Y | Chief   | Delete/destroy 6    | Filing  |
| Level 1       |      |   | of      | years, 3 months     | System  |
| Technical     |      |   | R       | after completion of |         |
| Program Plan  |      |   | esearch | grant/              |         |
|               |      |   |         | contract/agreement. |         |
|               |      |   |         | (5/37A)             |         |
+---------------+------+---+---------+---------------------+---------+
| Software      | N    | N | Chief   | Destroy when        | Filing  |
| Assurance NRA |      |   | of      | superseded,         | System  |
|               |      |   | R       | obsolete, or when 3 |         |
|               |      |   | esearch | years old,          |         |
|               |      |   |         | whichever is        |         |
|               |      |   |         | sooner.             |         |
|               |      |   |         |                     |         |
|               |      |   |         | (5/37C1)            |         |
+---------------+------+---+---------+---------------------+---------+
| Research      | Y    | Y | Chief   | Delete/destroy 6    | Filing  |
| Portfolio     |      |   | of      | years, 3 months     | System  |
| Plan          |      |   | R       | after completion of |         |
|               |      |   | esearch | grant/              |         |
|               |      |   |         | contract/agreement. |         |
|               |      |   |         | (5/37A)             |         |
+---------------+------+---+---------+---------------------+---------+
| Proposals     | N    | N | Chief   | Delete/destroy 6    | CIM     |
|               |      |   | of      | years, 3 months     | Tool    |
|               |      |   | R       | after completion of |         |
|               |      |   | esearch | grant/              |         |
|               |      |   |         | contract/agreement. |         |
|               |      |   |         | (5/37A)             |         |
+---------------+------+---+---------+---------------------+---------+
| OSMA SARP     | Y    | Y | Chief   | Delete/destroy 6    | Filing  |
| Level 2       |      |   | of      | years, 3 months     | System  |
| Technical     |      |   | R       | after completion of |         |
| Program Plan  |      |   | esearch | grant/              |         |
|               |      |   |         | contract/agreement. |         |
|               |      |   |         | (5/37A)             |         |
+---------------+------+---+---------+---------------------+---------+
| Grants and    | N    | N | Chief   | Delete/destroy 6    | Filing  |
| Contracts     |      |   | of      | years, 3 months     | System  |
|               |      |   | R       | after completion of |         |
|               |      |   | esearch | grant/              |         |
|               |      |   |         | contract/agreement. |         |
|               |      |   |         | (5/37A)             |         |
+---------------+------+---+---------+---------------------+---------+
| Research      | Y    | Y | Chief   | Delete/destroy 6    | CIM     |
| Initiative    |      |   | of      | years, 3 months     | Tool    |
| Record in the |      |   | R       | after completion of |         |
| CIM Tool      |      |   | esearch | grant/              |         |
|               |      |   |         | contract/agreement. |         |
|               |      |   |         | (5/37A)             |         |
+---------------+------+---+---------+---------------------+---------+