# Presentation: 810410

## Pre-Software Assurance Symposium Facility Initiatives Briefing

**Independent Verification & Validation **
**of **
**Programmable Logic Devices**

- 8 July 2005
- NASA IV&V Facility
- James Cercone, Ph.D., P.E.,WVU-Tech
- Michael Beims, SAIC

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Pre-Software Assurance Symposium Facility Initiatives Briefing

-   Review IV&V of PLD Research Project Objectives and  Framework
-   Review of detailed technical findings and VHDL defect taxonomy
-   Provide overview of Work Instruction development

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Pre-Software Assurance Symposium Facility Initiatives Briefing**

## IV&V PLD Status

**NASA-STD-8739.8**
*Software V&V is concerned with ensuring that software being developed or maintained satisfies  functional and other requirements and that each phase of the development process yields the right  products.*
*...**.. IV&V is performed by an organization that is technically, managerially, and financially  independent of the development organization.  For NASA, IV&V is performed and/or managed by  the NASA IV&V Facility.*
*...“**Software includes programs and operational data contained in hardware (e.g. ****firmware, programmable logic, and programmable gate arrays****).” *

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**IV&V PLD Status**

## IV&V PLD Status

**IEEE STD 1012-1998**
-  *IEEE Standard for Software Verification and Validation, provides supporting information regarding the integration of IV&V into every step of the Software Development Life Cycle. The IEEE standard, like the NASA Standard, also cites firmware and microcode in its definition of software: ***“This standard applies to software being developed, maintained, and reused .... The term Software also includes firmware, microcode, and documentation.” **

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**IV&V PLD Status**

## IV&V PLD Status

**IEEE Std 1076TM-2002**
**Abstract:**
***VHSIC Hardware Description Language**** (VHDL) is defined. VHDL is a formal notation   intended for use in all phases of the creation of electronic systems. Because it is both machine readable and human readable, it supports the development,*_*** verification***_*, synthesis, and testing of hardware designs; the communication of hardware design data; and the maintenance, modification, and procurement of hardware. Its primary audiences are the implementers of tools supporting the language and the advanced users of the language*.
**Keywords:**
*computer languages, electronic systems, hardware, hardware design, VHDL*

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**IV&V PLD Status**

## NESC Project Activities performed by IV&V

| From Project Plan (SAIC Document #ISTO-05-98-192), Section 3 – Activities | Completion Date |
| --- | --- |
| Identify the FPGA design logic faults from:
  NASA and industrial sites,
  Document Artifacts, and
  Comparison of typical FPGA logic design methods with proven software engineering methodologies, including those used for design and peer review | Year 1, 2Q |
| 2. Identify existing software engineering methodologies that can be directly applied to FPGA designs by tracing common defects to their underlying cause | Year 1, 3Q |
| 3. Suggest enhancements to developers’ design and peer review methodologies | Year 2, 1Q |
| 4. Provide field prototyped training materials for performing PL software V&V | Year 2, 4Q |
| 5. Successfully complete a pilot project | Year 2, 4Q |

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Primary Goals: **
**  ****Develop an IV&V strategy for PLD’s**
**  ****Provide field proven PLD Work Instruction (WI) to the IV&V practitioner**

## Activities Thus Far
Better understanding of PLD’s at WVU Tech and SAIC
Primarily via literature searches and attendance at workshops
Presentation to IV&V CAWG
WVU Tech obtained and learned IDE’s (Integrated development   Environment) for both Actel and Xilinx PLD’s. WVU Tech has also obtained and learned Active HDL

Limited analysis and simulation of NASA project data
IV&V has mapped PLD’s into a better framework for IV&V WI development
Identifying a taxonomy of defects in VHDL Domain
Via Literature Search
By comparing VHDL releases for the same chip
Evaluating SW code defects that can be “mapped” to PLD VHDL defects
Had initial discussions with JWST as candidate for VHDL Pilot Project

- Better understanding of PLD’s at WVU Tech and SAIC
- Primarily via literature searches and attendance at workshops
- Presentation to IV&V CAWG
- WVU Tech obtained and learned IDE’s (Integrated development   Environment) for both Actel and Xilinx PLD’s. WVU Tech has also obtained and learned Active HDL
- Limited analysis and simulation of NASA project data
- IV&V has mapped PLD’s into a better framework for IV&V WI development
- Identifying a taxonomy of defects in VHDL Domain
- Via Literature Search
- By comparing VHDL releases for the same chip
- Evaluating SW code defects that can be “mapped” to PLD VHDL defects
- Had initial discussions with JWST as candidate for VHDL Pilot Project

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Activities/Results thus far ****(9-1-05 through 8-8-05)**

## Activities/Results thus far (9-1-05 through 8-8-05)

- _Results Thus Far_
- Scoped PLD IV&V Framework to understand 05 accomplishments and identify potential future year activity
- Based on increased understanding, and
- Realization that existing IV&V Code Analysis WI is insufficient for PLD analysis
- Defect taxonomy, in process of refinement, for presentation at MAPLD

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Development Environments

- _Idealized Software/PLD Development_

- Requirements

- Design

- Code

- Test

-  SRS is a CM’d
-   document
-  Rigorous flowdown
-    is common

-  SDD is a CM’d
-    document
-  Manual or tool
-    generated

-  Different types of
-    code (C, C++, etc)
-  Mature tools avail to
-    aid in development,
-    verification

-  Performed on
-    implemented code
-  Unit, Subsystem,
-    System testing follows
-    req’ts flowdown
-  Rigorous process

- _Common PLD Development_

- Requirements

- Design/code
- and simulate
- at functional
- level

- Design/code
- and simulate
- After chip
- layout

- Testing after
- PLD is
- programmed

-  Part of subsystem
-  Hardware artifact
-    (e.g. EQ spec,
-    product functional
-    spec)

-  Performed on
-    implemented PLD
-  Unit, Subsystem,
-    System testing follows
-    req’ts flowdown

* **It is at this stage that Idealized/Common development *
*   **processes diverge*
*Design process*
*IDE*
*Target*
* **PLD’s also have timing concerns that are rare in software development, such as*
*Synthesized versus native components*
*Race Conditions*
*Adequately buffering data*

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Current Year (2005) Activities against PLD IV&V Framework

- Our current WI activity focuses on verification of VHDL Design
- Develop Work Instruction
- Flesh out Work Instruction with Pilot Project
- Deploy Work Instruction at Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Ideas for Future Year Projects

- Future Projects
- Develop WI for verification of Verilog or Schematic designs (item “a”)
- Provide WI for Requirements Analysis or Test Analysis of PLD’s (item “b”)
- This is probably a straightforward extrapolation of current WIs for FSW
- Have the breadth of knowledge on development tools and all target PLD’s (item “c”)
- Provide insights on timing/validation aspects of PLD implementation (item “d”)

- d

- b

- b

- a

- c

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Complexity is a Challenge for all Design Representations

**Complexity is a Challenge for all Design Representations**

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

- IV&V Facility

**Functional Trace / Performance Test**

**Design Trace / Functional Test**

**?**

## Review of detailed technical findings 

- and
- VHDL defect taxonomy

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Pre-Software Assurance Symposium Facility Initiatives Briefing**

## 
Entity

Are Signals defined in the port list as out type signals given values?

Are Signals that are defined in the port list as inout type signals used for both 
reading and writing?

Are Signals defined in the port list as in type signals used in the architecture?

**Entity**
- Are Signals defined in the port list as **out** type signals given values?
- Are Signals that are defined in the port list as **inout** type signals used for both
- reading and writing?
- Are Signals defined in the port list as **in** type signals used in the architecture?

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Potential VHDL“Hot Spots” visible in semantics**

## 
Process		

	Is there a series of sequential statements followed by a branching structure?

	Is there a branching structure followed by a series of sequential statements?

	Is each process sensitive list made up of the signals from the Entity’s port list?

**Process**
- 	Is there a series of sequential statements followed by a branching structure?
- 	Is there a branching structure followed by a series of sequential statements?
- 	Is each process sensitive list made up of the signals from the Entity’s port list?

**Sample Findings**** ****Potential VHDL“Hot Spots” visible in semantics**

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## 
If Structures

	Having elsif and no else statement

	Having neither an elsif or else statement

	Is there unreachable code inside an else statement?

	When using a compound if statement, are all possible conditions covered in subsequent elsif and else statements.

	How deep is the nesting of if structure?

	Testing Signals in the condition that are not part of the process’s sensitive list

**If Structures**
- 	Having elsif and no else statement
- 	Having neither an elsif or else statement
- 	Is there unreachable code inside an else statement?
- 	When using a compound if statement, are all possible conditions covered in subsequent elsif and else statements.
- 	How deep is the nesting of if structure?
- 	Testing Signals in the condition that are not part of the process’s sensitive list

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Potential VHDL“Hot Spots” visible in semantics**

## 
Signal Assignment

	Is the same set of Signals assigned values in each of the if-elsif-else sections?

	Is the same set Signals assigned values in each of the case structures 

when and when others => clauses?

	Are all Signals in a component’s port list mapped values during a 

Component’s instantiation?

**Signal Assignment**
- 	Is the same set of Signals assigned values in each of the if-elsif-else sections?
- 	Is the same set Signals assigned values in each of the case structures
- when and when others => clauses?
- 	Are all Signals in a component’s port list mapped values during a
- Component’s instantiation?

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Potential VHDL“Hot Spots” visible in semantics**

## Sample Findings Static Metrics Analysis of Public  Code 

- Sample Findings Static Metrics Analysis of Public  Code** **

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Sample Findings Static Metrics Analysis of Public  Code 

**Note !**

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Static Metrics Analysis of Public  Code**

## Sample Findings Static Metrics Analysis of Public  Code 

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Static Metrics Analysis of Public  Code**

- ...

## Sample Findings Static Metrics Analysis of Public  Code 

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Static Metrics Analysis of Public  Code**

**Observations related to Code Changes**
- (absolutes for the public code analyzed)
- A code change occurred if:
- There were 12 or less files
- Average number of lines per file was greater than 400
- There were less than 3 “package” statements
- There were less than 11 “entity” statements
- There were less than 20 “component” statements
- There were less than 13 “architecture” statements
- There were less than 20 “clk’event” statements
- There were any “while” statements
- There were any “wait” statements
- There were any “after” statements

## Sample Findings Static Metrics Analysis of Public  Code 

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Static Metrics Analysis of Public  Code**

**Observations related to Code Changes**
- (normalized for the public code analyzed)
- A code change occurred if:
- 5% or more of the total lines were “signal” statements
- 5% or less of the total lines were “in” statements
- 5% or less of the total lines were “out” statements
- 31⁄2% or more of the total lines were “if” statements
- 1⁄4% or more of the total lines were “case” statements

## Sample Findings Static Metrics Analysis of Public  Code 

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Static Metrics Analysis of Public  Code**

## Sample Findings Static Metrics Analysis of Public  Code 

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Static Metrics Analysis of Public  Code**

## Sample Findings Static Metrics Analysis of Public  Code 

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Static Metrics Analysis of Public  Code**

## Sample Findings Sample taxonomy of semantic defects visible in VHDL

**Sample Findings**** ****Sample taxonomy of semantic defects visible in VHDL**

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Sample Findings Sample taxonomy of semantic defects visible in VHDL

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Sample taxonomy of semantic defects visible in VHDL**

## Sample Findings Sample taxonomy of semantic defects visible in VHDL

- Note: NASA experts’ recommended practice prevents an ‘out of sync’ by insuring that resets are never very close to a clock edge.  This design is seen in NASA flight software as a D-FF with reset.

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Sample taxonomy of semantic defects visible in VHDL**

## Sample Findings Sample taxonomy of semantic defects visible in VHDL

***Synthesis vs Simulation difference visible in VHDL***
***Multiplexer with missing sensitivity signal (signal “b”)***

- process(a,sel)
-   if sel = '1' then
-      out <=1;
-   else
-      out <= b;
-   end if;
- end process

- www.synplicity.com/literature/pdf/HDLDesignMethods.pdf

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Sample Findings**** ****Sample taxonomy of semantic defects visible in VHDL**

## Fault Detection Matrix

|               Actual

Tested | False | True |
| --- | --- | --- |
| Positive | False/Positive
Potential Hot Spot identified/ 
No defect | True/Positive
Potential Hot Spot identified/ 
Design defect exists |
| Negative | False/Negative
No Hot Spot identified/ 
Defect exists | True/Negative
No Hot Spot identified/ 
No Defect  |

- High Confidence – IV&V success

- Less Confidence -  Mission Risk

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Fault Detection Matrix
Examples

_*True/Positive*_  *Potential Hot Spot identified/ Design defect exists*
***“****Pre-processor directives and usage are often not controlled by coding standards.*
*1) #ifdef statements can be left active or inactive and permit non-flight code (e.g. test code) to be compiled. Instances of such errors have been found in Mars program code.*
*2) #define statements can be left in the code from testing leaving test values or conditions active in the flight code. Instances of such errors have been found in Mars program code****.”***

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Fault Detection Matrix**
**Examples**

## Fault Detection Matrix
Examples

**Examples**

- process (A,B)
- begin
-    if (cond1)
-        X <= A + B;
-    elseif (cond2) X <= X – B;
-    end if;
- end process;

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

- If neither cond1 nor cond2 is true,  then X will  retain its value ...basically, X is stored in a latch
- In general, latches are not usually recommended in synchronous  designs

_*False/Negative  *_*No Hot Spot identified/  Defect exists*

## Taxonomy of Common Visible Defects

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## VHDL Code Severity Chart

| Severity | Description |
| --- | --- |
| A | FPGA code will not perform desired tasks.  Mission is jeopardized. |
| B | Serious hindrance to mission accomplishment.  A serious cost effect to project. I.E. (No “quick fix”) |
| C | Adversely affects FPGA code performance.  A minimal cost effect to project but a “quick fix” is possible. |
| D | Annoying effect to user but FPGA code is operable. |
| E | Any other effect. |

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Overview of Work Instruction Development

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Pre-Software Assurance Symposium Facility Initiatives Briefing**

## Work Instruction Development

**Work Instruction Development**

**Background Considerations **
- VHDL specific considerations
- Taxonomy of potential “Hot Spots”
- 	Clock and Reset Lines
- 	Sensitivity Lists
- 	Features not consistently supported between IDE’s
- 	Non-implemented features (i.e some attributes)

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## VHDL Code Severity Chart

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## VHDL Code Severity Chart Examples

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Example of High Functional Criticality -**
-  The FPGA with this defect is the only functional capability for the Satellite to deploy the solar panels.  If the FPGA does not perform this function, the satellite will run out of power, causing loss of the mission.

## VHDL Code Severity Chart Examples

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Example of Low Functional Criticality -**
- The FPGA controls one side of a dual redundant path to the telemetry transponder.  If the FPGA fails, then the telemetry is routed via CPU control, causing a momentary delay in telemetry, if all telemetry is buffered through on-board storage.  (Note: since this is a design (software) defect, if there were two identical FPGA's controlling this functionality, instead of an FPGA and the CPU, then the redundant FPGA can be expected to fail in the same manner and there is no functional redundancy, making this a High Functional Criticality defect.)

## Example of Vendor Specific Degree of Compliance 

“major differences between XVHDL and Express is IEEE VHDL-93 compliance. XVHDL is a fully IEEE VHDL-93 compliant tool. Express supports many of the most commonly used VHDL-93 synthesis constructs, but is not yet fully compliant; it remains officially compliant with the IEEE VHDL-87 standard.” 

http://www.xilinx.com/xlnx/xil_ans_display.jsp?iLanguageID=1&iCountryID=1&getPagePath=5144   (7/21/2005)

- “*major differences between XVHDL and Express is IEEE VHDL-93 compliance. XVHDL is a fully IEEE VHDL-93 compliant tool. Express supports many of the most commonly used VHDL-93 synthesis constructs, but is not yet fully compliant; it remains officially compliant with the IEEE VHDL-87 standard.”*** **
- _[http://www.xilinx.com/xlnx/xil_ans_display.jsp?iLanguageID=1&iCountryID=1&getPagePath=5144](http://www.xilinx.com/xlnx/xil_ans_display.jsp?iLanguageID=1&iCountryID=1&getPagePath=5144)__   (7/21/2005)_

**Compliance Issues**

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Example of Vendor Specific Compliance 
(two examples)

Signal Declaration
Supported ("register" or "bus" type signals are not supported) 

Attribute
Only supported for some predefined attributes: HIGH, LOW, LEFT, RIGHT, RANGE, REVERSE_RANGE, LENGTH, POS, ASCENDING, EVENT, LAST_VALUE
Otherwise, ignored. 

http://www.xilinx.com  (7/21/2005)

- (two examples)
- Signal Declaration
- Supported ("register" or "bus" type signals are not supported)
- Attribute
- Only supported for some predefined attributes: HIGH, LOW, LEFT, RIGHT, RANGE, REVERSE_RANGE, LENGTH, POS, ASCENDING, EVENT, LAST_VALUE
- Otherwise, ignored.
- _[http://www.xilinx.com ](http://www.xilinx.com/xlnx/xil_ans_display.jsp?iLanguageID=1&iCountryID=1&getPagePath=5144)__ (7/21/2005)_

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Compliance Issues**

## Defect Taxonomy (VHDL Verification at      Functional Design), and Pilot Deployment

- Draft of VHDL programming standards geared toward defect identification
- Defects commonly detected by compilers are not included
- Includes syntax, timing margins, clock boundaries
- This draft is in process of update, peer review
- Align defects with known coding defects
- Test draft product against actual VHDL text
- Developer places multiple revs of VHDL on website
- The results will be presented at MAPLD in September, 05

- Note: GLAST LAT used Actel VHDL for design and this served as basis for IR&D project.
- MRO project used Xilinx Verilog for design.

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

- Work Instruction in 05 addressed VHDL verification. Future tasks needed to address:
- Verilog and Schematic verification plus validation.

## Preliminary Considerations for Defect Detection in VHDL Based Designs

- Materials needed to start the verification process are:
- Design Documentation to analyze performance
- Actual VHDL Code
- Code Pedigree
- (Reused modules, designers, level of experience...)
- Development and Analysis Tools State diagrams.
- Clock Trees
- NASA and IEEE Standards

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## V&V Process and Procedure at the Code Level

Static Metric Analysis
Code Coverage (particularly for behavioral level designs)
Verification of Clock and Reset Tree’s (if provided)
Check for compliance to NASA Standards
Check for device resource usage 
(synthesized vs. board components such as MAC’s, SR, and DFF’s)
Check of IDE specific restrictions 
Check VHDL specific “Hot Spots”

- Static Metric Analysis
- Code Coverage (particularly for behavioral level designs)
- Verification of Clock and Reset Tree’s (if provided)
- Check for compliance to NASA Standards
- Check for device resource usage
- (synthesized vs. board components such as MAC’s, SR, and DFF’s)
- Check of IDE specific restrictions
- Check VHDL specific “Hot Spots”

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Preliminary Considerations for Defect Detection in VHDL Based Designs**

## Pre-Software Assurance Symposium Facility Initiatives Briefing

-   Review IV&V of PLD Research Project Objectives and  Framework
-   Review of detailed technical findings and VHDL defect taxonomy
-   Provide overview of Work Instruction development

- IV&V Facility

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

**Pre-Software Assurance Symposium Facility Initiatives Briefing**

## Background Slides

## Independent
Technical: IV&V prioritizes its own efforts
Managerial: Independent reporting route to NASA Headquarters
Financial: Budget is allocated by NASA to the IV&V Facility such that IV&V effectiveness is not compromised

Verification (Are we building the product right?)
The process of determining whether or not the products of a given phase of the software development lifecycle fulfill the requirements established during the previous phase
The product is internally complete, consistent and correct will support the next phase

Validation (Are we building the right product?) 
The process of evaluating software throughout its development process to ensure compliance with software requirements. This process ensures:
Expected behavior when subjected to anticipated events
No unexpected behavior when subjected to unanticipated events
System performs to the customer's expectations under all operational conditions

- Technical: IV&V prioritizes its own efforts
- Managerial: Independent reporting route to NASA Headquarters
- Financial: Budget is allocated by NASA to the IV&V Facility such that IV&V effectiveness is not compromised
_***Verification***_ (Are we building the product right?)
- The process of determining whether or not the products of a given phase of the software development lifecycle fulfill the requirements established during the previous phase
- The product is internally complete, consistent and correct will support the next phase
_***Validation***_ (Are we building the right product?)
- The process of evaluating software throughout its development process to ensure compliance with software requirements. This process ensures:
- Expected behavior when subjected to anticipated events
- No unexpected behavior when subjected to unanticipated events
- System performs to the customer's expectations under all operational conditions

- What is IV&V ?

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Maximizing Project V&V and IV&V

- The Project V&V goes end-to-end
- Needs sufficient depth to help ensure that they have build the product right and built the right product
- IV&V needs to be the second line of defense
- Select the most critical functionality and then IV&V to the appropriate depth -- not exceeding the IV&V point of diminishing returns (maintaining reasonability)
- The cut-off point should be where we have found the critical defects and also gained enough confidence in the software to support mission assurance requirements and launch recommendations
- Project Teams should compare our criticality rankings to their knowledge of the development as an independent source and explore differences
- Project Teams should look at activity just below the IV&V line to ensure adequate V&V resources

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## PLD Updated Framework

- Framework to perform IV&V was updated to
- Take into consideration that development of PLD’s is different than development of software,
- Address verification and validation tasks explicitly
- PLD’s are developed in an environment that combines design, code and simulation simultaneously
- Timing aspects much more critical in PLD’s
- Major differences needed to be addressed:
- PLD development is part of subsystem development – comparable artifacts not consistently generated
- PLD design can occur in many forms
- Schematic (representation similar to chip/board design)
- VHDL (representation similar to Ada)
- Verilog (representation similar to C/C++)
- Target system matters
- Syntax different even when same language used
- Development environment matters
- From a syntax standpoint
- From a capability standpoint (e.g. software motivated or hardware motivated)
- Which revision of IDE (multiple releases for hw motivated IDE’s)

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Updated IV&V Framework for PLD Development

- The above updated IV&V framework has more detail
- Allows us to clearly understand what we have accomplished, and what lies ahead
- Strategy is to perform accomplished tasks well
- For each task performed,
- Develop Work Instruction (WI), flesh out internally
- Test on pilot project
- Deploy updated WI
- The fast pace of PLD product evolution requires additional considerations
- Important to have cognizance of market trends
- Update WI appropriately with trends that will be implemented in spacecraft in the near term

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

## Simulink  Example

**Simulink  Example**

- http://www.transtech-dsp.com/software/simulink.asp

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

- IV&V Facility

## Small Microprocessor Example

**Small Microprocessor Example**

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

- IV&V Facility

## Examples of Rare Software Defects

- July 8, 2005

- IV&V of Programmable Logic Devices – Beims, Cercone

- IV&V Facility

*PLD’s also have timing concerns that are rare in software development, such as:*
*Synthesized versus native components – Seen in Stinger real time seeker simulations.  Debate over whether a native compiler matrix multiplication routine was sufficiently predictable versus a matrix multiply built up in separate Fortran 77 instructions.*
*Race Conditions – Seen in the “Response to a Setting Satellite Vehicle” scenario in the Space Shuttle GPS’ firmware.*
*Adequately buffering data – Seen in the Space Shuttle Primary Avionics Software Systems’ Mid Frequency Executive where every variable must be analyzed for time homogeneity and treated accordingly.*
- In General, “PLD–like” defects are seen in hard, real time software systems, which in turn are the primary candidates for migration to PLD’s in the near future (recent past?)