# Presentation: 249802

## ITS/CVO Architecture Conformance:Interoperability Testing StrategyTable of Contents

- 6.1  Steps
- 6.2  Prerequisites
- 6.3  Logistics and Mechanics
- 6.4  Configuration Management
- 6.5  Re-tests

*Table of Contents*

- ITS/CVO Architecture Conformance:Interoperability Testing StrategyTable of Contents

- 1	Introduction
- 2	Terminology
- 3	Scope
- 4	Roles and Responsibilities
- 5	Test Environment and Tools
- 6	Test Process
- 7	Key Criteria
- Appendix A  Level 1 Interoperability Test Matrix
- Appendix B  Example of Interoperability Testing Status

## Pairwise Interface Testing The Test Facility as a Substitute for One End of a Pair of Products That Interact According to a Standard

- 1	Identify each pair of products that interact using standards.
- 2	Review the standards.
- 3	Review test scenarios defined for MD/VA product testing.  Select test scenarios that test each aspect of the standard, focusing first on nominal interaction between the products.
- 4	Review test cases defined for MD/VA testing.  The interoperability test cases should be a subset.
- 5	Generate interoperability test data, using MD/VA test data as a start.  Cover nominal data, erroneous conditions, and extraordinary data.
- 6	Tailor the test data as needed.
- 7	Verify that the product has been successfully tested in stand-alone fashion and can generate/use the data intended for the interface testing.
- 8	Connect the product to the test facility and exchange the pre-defined data from one to the other according to test cases.  Test the inputs to and outputs from the product.   Verify that the product generates its expected outputs by executing tests, capturing the outputs, and verifying that they match the interface agreements.  Verify that the transmission mechanism did not distort the data.  Verify that selected outputs from one product into the other are those expected.   Execute all test cases defined.

*more. . .*

*Steps*

- APL tasks

- Product owner tasks

## End-to-End Interface Testing The Test Facility as a Substitute for One or More “Jurisdictions” (State, Carrier, Core Infrastructure) 

- 1	Identify** **the end-to-end processes that require testing.
- 2	Identify the systems involved.
- 3	Verify that the systems have been tested.
- 4	Review test scenarios defined for MD testing.  Select test scenarios that exercise each function in normal operational configurations.
- 5	Review test cases defined for MD testing.  The interoperability test cases should be a subset.
- 6	Generate interoperability test data, using MD test data as a start.  Cover nominal data, erroneous conditions, and extraordinary data.
- 7	Tailor the test data as needed.
- 8	Run the tests using standard operating practices, if possible.  Verify that each product/system receives the data expected from the other products/systems, that the data are interpreted correctly, and that the correct results are generated.  As necessary, substitute the test facility for one or more products/systems to avoid disrupting normal operations in that product/system.
- 9	Verify that no unintentional changes to the data occur, and that conversions/translations performed in each product/system yield accurate results.
- 10	Repeat, covering a range of data possibilities, including error conditions in various products/systems and user inputs, and all data sets in the test suite.

*Steps*

- APL tasks

- System owner tasks

## Prerequisites for Pairwise Interface Testing

- Each product under test, and required ancillary products, should be fully tested.
- Data should be controlled by the tester and/or the test environment.
- Any special test code needed to verify inputs and outputs should be fully tested.

*Prerequisites*

## Prerequisites for End-to-End Interface Testing

- Each pair of products involved in the end-to-end process should have been successfully integrated and tested.
- Data should be controlled by the tester.
- As much as possible, the tests’ operating environment should be the real operating environment, with no special test code inserted into the systems under test.

*Prerequisites*

## Interoperability Testing is Accomplished in Stages to Avoid Disrupting Operations 

- Interoperability Testing is Accomplished in Stages to Avoid Disrupting Operations

- Product A
- (e.g., IDT CAT)

- Product B
- (e.g., MD CI)

- Stage 1
- Pairwise Interface Tests

- Stage 2
- End-to-End Interface Tests

- Product D
- (MD CVIEW)

- Product A
- (IDT CAT)

- Product B
- (MD CI)

- Product E
- (SAFER)

- Product F
- (MD Roadside
- Ops)

- Objective:  Verify that interfaces between selected pairs of products/ systems meet the applicable standards

- Objective:  Verify dataflow and data usage among several selected products/systems meet the applicable standards

*Logistics & Mechanics*

- Product C
- (MD Legacy)

## Logistics & Mechanics

- Product/system owner contacts APL/Brenda Clyde four weeks in advance of projected test date to schedule the facility, identify the POC, and request test documentation.
- APL delivers test documentation.
- Mostly on CD or other removable media
- May also include some paper documents
- May be delivered via the CVISN Web Site:  *www.jhuapl.edu/cvisn/*
- APL establishes user account and provides to POC.
- POC provides evidence that prerequisites have been met two weeks in advance.

*Logistics & Mechanics*

## Configuration Management is Critical to the Interoperability Testing Process

- Each product or system subject to interoperability testing must be under configuration control, and provide evidence that the plans/procedures are being followed.
- The test facility, tools, and test documentation must be under configuration control.
- During the test conduct, any changes to the configuration of the products or systems under test, system test environment, test facility, test tools, and test documentation must be documented according to the CM policies and all parties involved in the test informed.

*Configuration Management*

## Re-tests are Required in Certain Circumstances

- In the event of a failure or other test problem, a retest may be requested within the schedule constraints of the testers and the facility.
- Retesting is expected to be requested.
- To verify that a problem is repeatable
- To demonstrate a successful fix after implementing change(s) according to CM policies
- Retesting should **not** ordinarily be requested to help troubleshoot the problem.

*Re-tests*

## Slide 10