# Presentation: 053785

## Lessons Learned from Encoding the DON NDR 

- March 15, 2006
- US/DoC/NIST/MEL/MSID

## Outline of the Talk

- Who we are
- What we are doing:  NDR encoding project
- What we’ve learned
- Recommendations
- Demonstrations:
- Quality of Design Tool
- Validation Page
- Where we are going
- Discussion

**Notes:**
The purpose of this slide is to tell the audience what we want to tell them and to hint that the proposed MDLC, the IDEF model is not an absolute model. However, it serves our purpose to motivate our research work around MDLC to help the industry achieve interoperability.

## Who We AreUS/DoC/NIST/MEL/MSID

- United States Government
- Department of Commerce
- National Institute of Standards and Technology
- Manufacturing Engineering Laboratory
- Manufacturing Systems Integration Division
- = A government resource that provides technical solutions to advance system integration capabilities.

## Slide 4

## Background

- NIST B2B Interoperability Testbed
- XML-based interoperability project with the automotive and aerospace industries
- NIST AEX Testbed
- XML-based interoperability project with the building construction industry
- Product Data Exchange and Validation Testing activities
- Data exchange-based integration project, not XML-based (ISO10303: STEP)
- Common characteristics
- Specifications are segmented
- Data exchange specifications evolve as integration projects proceed

**Notes:**
hide

## Content Standards are Bridges 

- Content Standards are Bridges

## Bridge Building Process

- Design the bridge
- Test the design
- Build a frame
- Pour the concrete
- Paint the roads
- Stress test
- Maintenance

## NDRs are a Basis for Building Interoperability Bridges

## Rationale for Rules

***Clarity****:  *make the semantics of a construct clear to the user
***Structural clarity****: *contribute to a schema’s readability, which can facilitate consistent interpretation of a standard
***Extensibility****:  *promote reuse through extension
***Common symbolic syntax****:  *foster the use of common naming conventions. enable better automation and improve readability and clarity
***Maintainability****:  *reduce the maintenance burden especially when changes occur
***Performance****:  *reduce computational overhead associated with the XML instance parsing, validation, and other XML processing
***Interoperability****:  *promote interoperability among partners sharing the same schema
***Model validity****:  *ensure a schema’s semantic validity (e.g., no duplicate content)

## Benefits of NDR Encoding

- NDR can be enforced!*
- Encoding of rules results in better rules
- The process of encoding tests the rules
- Executing the rules results in better XML Schemas
- The benefits of the NDR are achieved
**Cycle time from requirements for a schema to production XML is reduced**
- * computationally enforced

## Experience From Korea

- Our partner, the Korean Institute of Electronic Commerce  (KIEC), has used our tool (QOD) to implement “Guideline for XML Document in Korea”
- Oct. 2005: tested 30 documents used in customs
- Feb. 2006: tested 29 documents used in electric power industry
- Feb. 2006: used in the development of documents  for port services area

## Lessons from Korean Experience

- Typically it takes 4 hours to examine a single electronic document manually
- Total validation time reduced from two weeks to 3 days (in power industry)
- Used as a supplementary tool during training on electronic document standardization
- Increased understanding of NDR
- Reduced errors in designing electronic documents

## Encoding DON NDR

- We encoded as many DON NDR rules as possible in Schematron
- NDR has 128 rules

## Problems In Encoding Rules

- Clarity of the rule  (text is not clear or text and examples conflict)
- External interfaces required
- Applies to something other than the schema
- Guidance rather than rule
- Mismatch with needs of technical publications

## Clarity

- MDC 3:  *If a DON ccts:DataType is extended or restricted, it MUST retain the original business context.*
- DateTypes in the ccts should be independent of business context
- NMS 2:  *URNs MUST be in lowercase, except multiple words, which MUST use lower camel case and the root element, which will use upper camel case.*
- NMS 11:  *All DON namespace declarations MUST be qualified.*
- The definition of *qualified* is left to user interpretation

## Requires External Interfaces

**GNR1:  ***DON XML element, attribute, and type names MUST be in the English language, using the Oxford English Dictionary for Writers and Editors (Latest Ed.). Where both American and English spellings of the same word are provided, the American spelling MUST be used*
- What is classified as the “Latest Ed.”
- No automated way to distinguish American from English spelling
- The Oxford English Dictionary does not offer an automated lookup service (last I checked.)

## Requires External Interfaces

**CDL 1**:  *All codes used in DON MUST be part of a DON-maintained or externally maintained Code List Schema module.*
*	*This rule would be testable if
- the lists exist and
- the interface for accessing the lists were known
- Full testing of this rule needs both these aspects.

## Not a Schema Design Rule

- ATD 4:  *The objectIDRef xsd:attribute value MUST be equal to the value of an ID ccts:BBIE element.*
- Applies to instance data rather than schema
- GRN 4:  *Abbreviations and acronyms MUST be submitted to an FNC for approval.*
- Applies to business process rather than schema design

## Guidance -- flaggable

**ELD 5**:  *For CCTS:BBIEs that are based on ID, code, and measure, a local element MAY be declared in the xsd:complexType of the parent ABIE.*
**GSX 8**: *The xsd:choice element MAY be used.*
- Use of the word MAY
- Pass/fail testing not possible
- Flagging may be possible
- Is the implication that use should be flagged or not?

## Guidance – not flaggable 

**CDL 2**:  *The DON library MAY design and use an internal code list if an existing external code list needs to be extended or if no suitable external code list exists.*
**GNR 6**: *DON XML element, attribute, and type names MUST be in singular form unless the concept itself is plural (example: goods).*
- Semantic understanding of words necessary
- CDL 2 could be flagged but the word *suitable* is subjective
- GNR 6 is not flaggable

## Mismatch with Needs of Technical Publications

*ELD6:  Empty elements SHALL NOT be declared except for reference elements and Xlink elements, which MUST be approved by the cognizant FNC and BSC*
- Rule is too restrictive
- EMPTY elements are traditionally used for graphics, multimedia, etc.
- People will circumvent this rule by creating a 'string' type and not plan on entering the data which will cause confusion on the user of the schema.

## Technical Publication Requirements

*MDC4:  **Mixed content MAY only be used when an XML schema component is defined by a namespace from a BSC-**a**pproved business standard (e.g. XHTML).*
- Mixed content is required for document-centric XML data.
- Identifying keywords and components in paragraph text is essential:
- Cross-references
- Superscript/subscripts
- In-line Graphics
- etx.c

## Lesson Learned in coding NDRs

- NDR documents need to be regarded as rigorous technical documentation
- Rules that can not be implemented are non-enforceable
- Definition of NDR’s is non-trivial
- Many rules can not be tested
- Many rules are more difficult to implement than thought
- Difficult to reuse rules due to namespace definitions
- Often rules are ambiguous or unclear
- Implementation of rules is non-trivial
- Testing of rules is complex
- All boundary conditions need to be thought of and covered
- Legacy data and 3rd party schemas need to be addressed in NDRs

## Recommendations for NDRG

- Define rules rigorously and include examples
- Identify actual external interfaces
- Limit scope of NDRG to XML Schema specification (define Dictionary requirements and process elsewhere)
- Distinguish rules from guidance for testability purposes
- Review requirements for technical publication and impact on data specification
- Partition NDR for data versus documents
- Recommend using QOD for developing NDR

## Schema Quality of Design Testing Tool (QOD)

- Contains rules based on naming and design guidelines (NDRs) from a number of sources
- Stores executable test cases written in Schematron and Java Expert System Shell (JESS)
- Executes tests against user-provided schemas and reports results
- Used to organize tests into profiles

**Notes:**
Surveyed naming and design rules documents
Coded some of the rules
Developed a harness for executing rules
Available as a web-based service

## Why QOD?

- Addresses proliferation of NDRs
- Overlapping NDR standards
- Supports reusability of rules
- Highlights ambiguous rules
- Provides an explicit structure for rules in NDRs
- Automates rule enforcement
- Enables versioning and traceability of rules

## Demo

- Quality of Design tool (QOD)
- Naming Report:  _[http://syseng.nist.gov/b2bTestbed/projects/xmlvalidation/NamingReport.html](http://syseng.nist.gov/b2bTestbed/projects/xmlvalidation/NamingReport.html....it)__ _
- Validation Page

## What’s Next for our Project

- Provide feedback to DON NDR authors
- Provide input into Federal NDRG effort
- Continue to expand our NDR rule-base
- Identify collaborators
- Develop NDR authoring environment
- Produce a *Toolkit *for XML Schema developers

## Collaboration Opportunities

- QoD tool and executable rules:  _[http://](http://qod.sourceforge.net/)__[qod.sourceforge.net](http://qod.sourceforge.net/)__[/](http://qod.sourceforge.net/)_
- Software enhancement based on user requirements:  Send us mail!
- _[xmlTestbed@cme.nist.gov](mailto:xmlTestbed@cme.nist.gov)_
- Tell us what you like
- Tell us what would make our tools more useful for your project
- Tell us if you want to help

## Future work:  NDR Authoring Environment

- Specify an XML Schema for NDRs
- Integrates schema for rule capture (QoD Schema) with documentation solutions
- Extend QoD Schema for rule capture to include documentation components of NDR
- Base on existing vocabulary such as DocBook or XHTML
- Use to connect rules to executable formats

## Benefits of NDR Authoring Environment

- Makes an NDR real
- Executable rules are integral to NDR
- Schemas can be tested for consistency with NDR
- Problems in written rules are flushed out as they are developed
- Reduce time spent on formatting
- Multiple outputs from same source (HTML, PDF, ...)
- Promote reuse
- Make it easier to extract rules from document
- Make it easier to tailor an existing NDR to meet new requirements
- Improve traceability

## Summary

- A process for XML schema development is necessary for promoting interoperability and reuse
- Tools can automate the schema development process thereby reducing labor and deployment time
- Definition and implementation of NDR’s is non-trivial but necessary to support reuse of schemas
- Use of NDRs will ultimately make XML schemas more acceptable to a wide audience
- Common format for NDR development, encoding of rules, and validation tools will increase the effectiveness of NDRs

## Contacts

- KC Morris – _[kcm@nist.gov](mailto:kcm@nist.gov)_
- Simon Frechette – simon.frechette@nist.gov
- Serm Kulvatunyou – _[serm@nist.gov](mailto:serm@nist.gov)_
- Josh Lubell – lubell@nist.gov
- Puja Goyal – _[pgoyal@cme.nist.gov](mailto:pgoyal@cme.nist.gov)_
- Betty Harvey, ECC, Inc. -- harvey@eccnet.com
- Websites
- Tools available from MSID: _[http://](http://www.mel.nist.gov/msid/xml_related.htm)__[www.mel.nist.gov/msid/xml_related.htm](http://www.mel.nist.gov/msid/xml_related.htm)_
- Developer’s site for QoD tool and executable rules:  _[http://](http://qod.sourceforge.net/)__[qod.sourceforge.net](http://qod.sourceforge.net/)__[/](http://qod.sourceforge.net/)_