# Presentation: 941376

## Overview of Silver Compatibility Reviews: What Reviewers are Looking For Robert Freimuth, Salvatore Mungal (VCDE)Tahsin Kurc (Architecture)

- ICR F2F Meeting
- Sept. 18, 2006

## Acknowledgements

- _VCDE:_
- Brain Davis (lead)
- Michael Keller (co-lead)
- _ICR:_
- Juli Klemm (lead)
- _Architecture:_
- Arumani Manisundaram (lead)
- Michael Keller (co-lead)

- _Semantic Review Training Team:_
- Salvatore Mungal
- Robert Freimuth
- Lewis Frey
- _Silver API Working Group:_
- Patrick McConnell
- Tahsin Kurc
- Avinash Shanbhag
- Moses Hohman
- John Osborne
- Vinay Kumar
- Andy Pople
- Michael Keller

## Agenda

- Overview of the Mentoring process and Silver Compatibility
- Overall Silver Compatibility Review Process
- VCDE criteria and process
- Architecture criteria and process
- Next Steps
- Q/A

## Compatibility Guidelines, Revision 2

- Interoperability can be defined as the ability of a system to _access_ and _use_ the parts of another system. The caBIG program has made interoperability between data and software components a primary strategic goal.
- The cross-cutting Architecture and Vocabulary/Common Data Elements Workspaces (VCDE) were created as part of the caBIG initiative ... for defining and ensuring interoperability.
- Silver Level Criteria: A rigorous set of requirements that, when met, significantly reduce the barrier to use of a resource by a remote party who was not involved in the development of that resource.

## The Mentoring Process & Compatibility Reviews

-   Mentors engaged “Early & Often”, preferably at project kick-off meeting
-   Mentors review deliverables to ensure project is moving toward silver compatibility
-   Mentors of a project serve as lead reviewer during compatibility review

## caBIGTM Compatibility (from caBIGTM Compatibility Guidelines Revision 2)

**Syntactic**

**Semantic**

**Semantic & **
**Syntactic**

- caBIGTM Compatibility (from caBIGTM Compatibility Guidelines Revision 2)

## http://gforge.nci.nih.gov/frs/download.php/457/Semantic_Integration_SOP-.3.zip

## Agenda

- Overview of the Mentoring Process and Silver Compatibility
- Overall Silver Compatibility Review Process
- VCDE criteria and process
- Architecture criteria and process
- Next Steps
- Q/A

## Review Workflow: Macro Process

## Review Workflow: Macro Process

## Review Workflow: Macro Process

## Review Workflow: Macro Process

## Agenda

- Overview of the Mentoring Process and Silver Compatibility
- Overall Silver Compatibility Review Process
- VCDE criteria and process
- Architecture criteria and process
- Next Steps
- Q/A

## VCDE Compatibility ReviewFrom the Developers’ Perspective

- UML modeling best practices
- Software tooling
- SIW
- CDE browser, UML Model browser
- Submission package
- Overview of checklist
- CDE standards
- CDE reuse
- Final report
- Resources and Links

## http://gforge.nci.nih.gov/frs/download.php/457/Semantic_Integration_SOP-.3.zip

## UML Modeling Best Practices

- Described in the caCORE SDK Programmers Guide, Chapter 5
- http://ncicb.nci.nih.gov/NCICB/infrastructure/cacoresdk
- ftp://ftp1.nci.nih.gov/pub/cacore/SDK/caCORE_SDK1.1_Programmers_Guide.pdf
- Questions?
- NCICB help desk (Claire Wolfe and colleagues)
- Architecture/VCDE mentors

## Examples of UML Modeling Best Practices(Chapter 5 of the Programming Guide)

- Allowable Relationships
- The SDK tools recognize association and generalization (inheritance) relationships.
- One word class names are capitalized
- One-word attribute names should be all lowercase
- Association End Navigability (Directionality)
- The navigability of each association end must be specified at both ends.
- Association End Role Names
- Both ends of each association must be given a role name
- Association End Multiplicity
- The multiplicity of each association end must be specified at both ends.

## VCDE Compatibility ReviewFrom the Developers’ Perspective

- UML modeling best practices
- Software tooling
- SIW
- CDE browser, UML Model browser
- Submission package
- Overview of checklist
- CDE standards
- CDE reuse
- Final report
- Resources and Links

## Automatic Error Checking Using the SIW

- SIW: Semantic Integration Workbench
- Automatic validation of the XMI file
- http://cadsrsiw.nci.nih.gov
- The first step of the VCDE review is to verify that the XMI file has an error-free SIW report

## Error-Free SIW Report

## CDE Browser and UML Model Browser

- Search for existing CDEs that could be reused
- http://cdebrowser.nci.nih.gov
- http://umlmodelbrowser.nci.nih.gov

## CDE Browser

## VCDE Compatibility ReviewFrom the Developers’ Perspective

- UML modeling best practices
- Software tooling
- SIW
- CDE browser, UML Model browser
- Submission package
- Overview of checklist
- CDE standards
- CDE reuse
- Final report
- Resources and Links

## Review Workflow: Macro Process

## Required Artifacts for the Submission PackageVCDE

- Brief description of the data system and its design (presentation to the VCDE and Arch WS)
- UML Model (diagram)
- Semantically annotated XMI file
- SIW report and clean error log
- UML Loader checklist (for registration in caDSR)
- Value Domain Report (permissible values, definitions)
- Vocabulary Report (controlled vocabularies used)

## Required Artifacts for the Submission PackageVCDE

- Standards Report (CDE and vocabularies used or relevant but not used)
- Full CDE Use Report (export from CDE Browser)
- Implementation/Design Document*
- Test Approach document*
- Use Case Documentation*

- *Optional (recommended to provide context)

## VCDE Compatibility ReviewFrom the Developers’ Perspective

- UML modeling best practices
- Software tooling
- SIW
- CDE browser, UML Model browser
- Submission package
- Overview of checklist
- CDE standards
- CDE reuse
- Final report
- Resources and Links

## Compatibility Review ChecklistOverview

- Information Models
- Classes sufficiently distinct for appropriate semantic integration
- Semantic annotation conforms to described data
- Followed by UML modeling best practices
- Data Elements
- All attributes correctly annotated and loaded into the caDSR
- Appropriate value domains curated
- Standard Data Elements used where appropriate
- Followed by ISO11179 best practices

## Compatibility Review ChecklistOverview

- Vocabularies
- Controlled Terminology used where appropriate
- Standard Vocabularies used where appropriate
- Followed by Vocabulary best practices
- Messaging Interfaces/APIs
- Now covered by the Architecture review

## Categories for Checklist Items

- Absolute
- Absolutely required for Silver Level
- All items must be satisfied
- Required
- Required as per the compatibility guidelines
- A few violations are permissible (at the discretion of the review group)
- Suggested
- Suggested best practices, items that improve interoperability
- Will never be used as a basis for failing a model

## Categorized Checklist ItemsExamples

- Absolute
- All classes and attributes must have descriptions
- Required
- Avoid using class identifiers in attribute names
- Suggested
- Class and attribute names are singular

## VCDE Compatibility ReviewFrom the Developers’ Perspective

- UML modeling best practices
- Software tooling
- SIW
- CDE browser, UML Model browser
- Submission package
- Overview of checklist
- CDE standards
- CDE reuse
- Final report
- Resources and Links

## CDE Standards

- Were CDE standards reused?
- Creation of CDEs in the model vs approval of the standard CDEs
- Standards located on GForge and caDSR
- http://gforge.nci.nih.gov/projects/cdestandards/
- http://gforge.nci.nih.gov/frs/?group_id=109 (“Files”)
- CDE browser (under “Data Standards”)

## Slide 34

## Accepted Data StandardsApproved by VCDE 

- OMB-derived Race and Ethnicity
- CDC-derived Race and Ethnicity Standard
- Representation of Date and Time for Info. Exchange
- Performance Status Scale
- Sex and Gender
- Person Name
- Organization
- Mailing Address
- Gene Identifier
- Language
- Marital Status
- Religion
- Age
- Social Security Number

## Future Data StandardsUnder Review or In Development

- Candidate Standards (under VCDE review)
- Telephone Number
- Units of Measure
- Proposed and Upcoming Standards (in development)
- Email address
- Organism
- Body Mass Index
- Body Surface Area
- Titles (name prefix, including civilian and military)
- Education levels
- Person Relationships (brother, sister, etc)

## VCDE Compatibility ReviewFrom the Developers’ Perspective

- UML modeling best practices
- Software tooling
- SIW
- CDE browser, UML Model browser
- Submission package
- Overview of checklist
- CDE standards
- CDE reuse
- Final report
- Resources and Links

## CDE Reuse

- Re-use a full CDE whenever possible
- Partial CDE reuse is better than no CDE reuse!
- CDE = 	Object (Class) +
- 			Property (Attribute) +
- 			Value Domain

## VCDE Compatibility ReviewFrom the Developers’ Perspective

- UML modeling best practices
- Software tooling
- SIW
- CDE browser, UML Model browser
- Submission package
- Overview of checklist
- CDE standards
- CDE reuse
- Final report
- Resources and Links

## Review Workflow: Macro Process

## The Final Report

## VCDE Review ReportMost Frequently Cited Comments

- Missing UML descriptions (SIW!)
- Inconsistent or ambiguous UML/CDE definitions
- CDEs do not have sufficient concepts for what they describe
- Lack of permissible values (enumeration)
- CDE reuse
- CDE standards
- existing CDEs from other models
- whole vs. partial
- Previous reviews are available at the Gforge site: http://gforge.nci.nih.gov/frs/?group_id=108

## VCDE Compatibility ReviewFrom the Developers’ Perspective

- UML modeling best practices
- Software tooling
- SIW
- CDE browser, UML Model browser
- Submission package
- Overview of checklist
- CDE standards
- CDE reuse
- Final report
- Resources and Links

## Resources

- VCDE Home Page
- https://cabig.nci.nih.gov/workspaces/VCDE/
- Compatibility Guidelines, Revision 2
- https://cabig.nci.nih.gov/guidelines_documentation
- caCORE SDK Programmers Guide
- ftp://ftp1.nci.nih.gov/pub/cacore/SDK/caCORE_SDK1.1_Programmers_Guide.pdf
- CDE Browser
- http://cdebrowser.nci.nih.gov/CDEBrowser
- UML Model Browser
- http://umlmodelbrowser.nci.nih.gov
- SIW
- http://cadsrsiw.nci.nih.gov
- caCORE and caDSR Training
- http://ncicb.nci.nih.gov/NCICB/training/cadsr_training

## ResourcesGforge Sites

- Silver Level Criteria Small Groups (VCDE and Architecture)
- http://gforge.nci.nih.gov/projects/silver-criteria/
- Arch-VCDE Compatibility Reviews
- http://gforge.nci.nih.gov/projects/compat-rev/
- Arch-VCDE Best Practices SIG
- http://gforge.nci.nih.gov/projects/archvcdebpsig/
- CDE Standards
- http://gforge.nci.nih.gov/projects/cdestandards/
- VCDE Mentors
- http://gforge.nci.nih.gov/projects/vcde-mentors/
- VCDE Liaisons
- http://gforge.nci.nih.gov/projects/vcde-liaisons/

## Agenda

- Overview of Silver Compatibility and the Mentoring process
- Overall Silver Compatibility Review Process
- VCDE criteria and process
- Architecture criteria and process
- Next Steps
- Q/A

## Architecture Compatibility Review-Overview

- Goal: 	Provide guidelines for mentoring and reviewing 	applications for silver-level compatible APIs
- _Step 1:_ Classify the system (or API) type
- _Step 2:_ Evaluate the API based on the system/API type
- _Step 3:_ Establish that the system exists
- Silver Level API Compatibility Document:
**Silver-level Compatibility Review Guidelines: Guidelines for Reviewing APIs**
- URL: http://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/silver-criteria/api/silver_compat_api_guidelines.doc?cvsroot=silver-criteria

## Types of APIs and Systems

- Client APIs
- Exposed directly to users through a programming language
- Messaging APIs
- Expose an API through a standard messaging protocol
- API is open to any client – language agnostic
- Preferred approach is to wrap a messaging API into a client API

## Types of APIs and Systems

- Data service (Data oriented system)
- Exposes a published Information Model
- Provides query method
- Query is an object
- Associations of objects returned should be “traversable” (from the returned objects or via further queries)
- Analytical services
- Algorithmic manipulation of input
- All needed associations of the input/output objects should be materialized
- Analytical tools
- Consume data registered in the caDSR via silver API or data source (e.g. file)
- Don’t need to produce data
- Command-line programs
- Should be wrapped into a client API

## Artifacts for review:General

- UML model
- Class diagrams for data model and API
- Programming language documentation
- Both API and data objects
- Examples: JavaDocs, PerlDocs
- Not needed for messaging APIs

## Artifacts for review: Existence

- Test script
- Demonstrates functioning API against real system
- Not expected that reviewer need to run the script
- Test log
- Log from running the test script
- Show methods, parameters, output

## Artifacts for review: Messaging APIs

- Messaging interface
- Public interface
- How to formulate and send messages
- Handshaking protocols
- Example: WSDL
- WS-I compliance recommended
- Electronic data format
- Describes structure of messages
- Example: XML Schema
- Also needed for tools that input files

## Architecture Compatibility Review: System/API classification

- Step 1: classify the System/API
- Determine type of system and API
- Developer helps
- Classification used to decide what criteria to apply when reviewing
- Inventory of artifacts should be made

## Architecture Compatibility Review: Data service API evaluation

- Step 2: evaluate a data service API
- Should provide one or more query methods
- A different query method should not be needed for each different class in the model
- API should not require knowledge of underlying persistent data source (e.g., relational database)
- API should enable querying of the published, exposed Information Model
- Query and tweaking parameters must be registered in the caDSR
- Returnable objects should be documented in API documentation or API should return typed objects
- API and objects documented should match the structure of the CDEs registered in the caDSR (can use UML)
- Recommended that model be reused from other project if it exists

## Architecture Compatibility Review: Analytical service API evaluation

- Step 2: evaluate an analytical service API
- Input and output object classes and parameters should be registered in the caDSR
- API and objects documented should match the structure of the CDEs registered in the caDSR (can use UML)
- Can consume/produce objects that could be considered information models by a data service
- Recommended that model be reused from other project if it exists

## Architecture Compatibility Review: Special case API evaluation

- Step 2: messaging interfaces
- Review message interface and electronic data format instead of programming language documentation
- Should still correspond to UML/CDEs
- Step 2: Errors
- Errors explicitly exposed by API should be registered in the caDSR

## Architecture Compatibility Review:Existence established

- Step 3: establish the existence of the system
- Review test script and log
- Data services
- Sufficient querying of classes, attributes, and associations should be demonstrated
- The reviewer should be confident that the information model is navigable
- Analytical services
- Invocation of a significant variety of Silver-level methods should be demonstrated
- Reviewer should be confident that the system is performing some algorithmic manipulation of data, and the data is structured as objects
- If reviewer is not satisfied by test script and log, the reviewer can ask the developer to set up the system so the reviewer can execute
- The test script and review the test log
- One or more additional test scripts

## Agenda

- Overview of Silver Compatibility and the Mentoring process
- Overall Silver Compatibility Review Process
- VCDE criteria and process
- Architecture criteria and process
- Next Steps
- Q/A

## Next Steps

- Architecture WS to vote on final version of Silver API Guidance White Paper
- VCDE WS to vote on final criteria checklists
- Use finalized documents in compatibility reviews
- Collect lessons learned from reviewers using criteria
- Update as needed

## Agenda

- Overview of Silver Compatibility and the Mentoring process
- Overall Silver Compatibility Review Process
- VCDE criteria and process
- Architecture criteria and process
- Next Steps
- Q/A