# Presentation: 845577

## DoD Metadata RegistryStatus

**Dr. Glenda Hayes**
**NCES Data Advisor**
**May 17, 2006**

- UNCLASSIFIED

**Notes:**
The DoD Net-Centric Data Strategy is the Department’s conceptual blueprint for achieving net-centricity – power to the edge.  NCES is an enabler for the COIs to use in achieving the goals laid out in that strategy.

## Agenda

**DoD Net-Centric Data Strategy**
**Data Strategy Implementation Actions**
**Role for NCES Enablers**
**DoD Metadata Registry**

## DoD Net-Centric Data Strategy

**Trusted**

**Interoperable**

**Responsive**

**Accessible**

**Understandable**

**Visible**

- Is an information resource discoverable by most users?

- Is it available on the network, and are tools readily available to use it?

- Can it be intelligibly used?  Are the semantics well documented?

- Can it be combined or compared with other information?  Can it be mediated?

- Is the source, accuracy and currency of the data available to users?

- Is the data what users need? Are robust user feedback mechanisms in place to improve it?

***The ***_***DoD Net-Centric Data Strategy***_*** is the plan to make the Department’s information resources:***

- CC/S/As must _institutionalize processes_ to accomplish these goals

**Notes:**
Given ubiquitous IP capable communications, the Department’s approach to net-centricity has been set forth in a documented called the DoD Net-Centric Data Strategy. It was signed out by ASD NII on 9 May 2003.  This strategy lays out the six clear-cut actionable goals shown here.  The questions beside each goal clarify their intent.  Progress toward these goals can be monitored and measured by observation of what capabilities are actually appearing online, whose using them, and what the users feedback to developers and operators. 

## Data Strategy Goals& Actions Required

- Implement DOD Data Strategy per Directive 8320.2

- Identify System Assets to Expose
- Determine Tagging Method & Technologies
- Maintain Catalogs
- Tag & Post Asset Info
- Register/Publish Metadata

* **NCES enables via online services, interface specs and *
*Software Development Kits*

- Identify Assets for Access
- Select Applications
- Provide Data Access Services
- Post Assets
- Register Service

- Select/Create Taxonomy & Vocabulary
- Develop Information Model & Register
- Develop data Payload packages & Register

- Create & Monitor Robust Feedback Mechanisms
- Closely link Development & Ops

- Provide Access
- Usage Controls
- Manage Rights
- Publish Data Asset Pedigree & Authoritative Source Designs

- Register Metadata
- Define & register Metadata Relationships
- Identify Key System Interfaces
- Comply with Net-Centric Interface Specs

**Data Asset Sponsors, Builders, and Operators: Specific Actions Required**

## Enable Data To Be Understandable

**	****“****Secure the Building!”****   What does it mean?**
_**Navy**_**: “Turn off the lights and lock the doors.” **
_**Arm**_**y: “Surround the building, occupy, and control entry.” **
_**Marine**_**s: “Call in close air support, assault with small team, neutralize occupants, fortify and hold at all costs until properly relieved. SEMPER FI!” **
***and...***
_**Air Force**_**:  “Take out a three-year lease with option to buy.” **

*One reason Government Agencies and Military Services have *
*trouble operating jointly is that they speak different languages. *

**Register Terms, Definitions, & Relationships**

## Support Data Interoperability

- Etc.

- Tomcat

- F-14

**Support Data Interoperability**

**Interoperability Requires:**
** *****Physical***** Identity **_***or***_** Translation/Transformation**
- Translate/Transform *via *neutral “mediation” schema
**(to reduce “*****n*****-squared” point-to-point interface problem)**

- F14

- F-14

- F/14

**Interoperable data thru “sameness”**
**Interoperable thru translation *****via***** neutral schema**
**Interoperable “pair-wise” *****but***** inefficient**

**Register Data Translations**

**Notes:**
But XML does not, by itself, solve any data sharing problems that plague the Department of Defense.  These alternate representations have persisted over time because there has been an inadequate appreciation of the interplay among persistent data such as that found in relational databases, message data, symbology, and other user interfaces.  Although a human operator can easily determine these are alternate terms for the same concept, a computer system requires sameness or mediation for interoperability.

## DoD Metadata Registry(MDR) and Clearing House

**“****One Stop” Publish, Subscribe and Manage capabilities for Reusable *****Structural***** Metadata Components **
**130k engineering artifacts (schema, stylesheet etc) registered as of 5/16**
**Includes translation logic for mediation & taxonomies **
**7k Registered Users & Systems from DoD, IC, DHS, NASA and more**
**Unclassified, Secret and Top Secret instances operating**
**Organized by “Gallery”**
**Metrics**
**Run Time support available **
**Web Services **
**ebXML capabilities**
_**User**_** feedback driven**
**Online user-to-developer**
**DoD Metadata Working Group**
**V5 just released**

- http://metadata.dod.mil

**Notes:**
The DoD Metadata Registry is the Department’s electronic marketplace for structural metadata components. It provides enterprise-wide visibility and access for developers, configuration controllers and program overseers to register, use, subscribe, and manage structural metadata.  The purpose of the Registry is not to publish broadly applicable standards, but to provide visibility into the Department’s ongoing structural metadata investment, and to encourage reuse of existing metadata components that are both developmental and operational.  While each registered component is identified with its status and distinguished by a namespace, the Registry’s purpose is to allow developers to pull whatever they need; not to try and madate usage of any particular metadata collection.  The benefits to the developer, reductions in cost, time savings and interoperability have proven to be sufficient incentive.  The public may access a portion of the DoD Metadata Registry via the Internet.  To examine metadata artifacts, users must be sponsored by a government employee.  The contents of the Metadata Registry are organized by governance namespaces within various metadata type “galleries.”  Communities of Interest (COIs) should align with and leverage collections within one or more namespaces that intersect their concerns.  

## MDR Accessibility 

**MDR Accessibility **

- Unclassified (Open)
**NIPRNET w/Internet Access**
_**[http://metadata.dod.mil](http://metadata.dod.mil/)**__** **_
- Unclassified (Sensitive)
**Open Registry contents *****plus***** **
**       “****password protected” components**
**NIPRNET with Internet Access**
_**[http://metadata.dod.mil](http://metadata.dod.mil/)**__** **_
- Secret
**Unclassified (Sensitive) content *****plus***** **
**       ****secret components**
**SIPRNET **
_**[http://diides.ncr.disa.smil.mil](http://diides.ncr.disa.smil.mil/)**_
- Top Secret (SCI)  Managed by ICMWG
**Secret content *****plus***** SCI components**
**JWICS**

**Unclassified (Sensitive)**

**Secret**

**Top Secret (SCI)**

**Unclassified (Open)**

***Users Discover and Pull metadata they are cleared for -***

**Synchronized Components on Multiple Networks**

- Engineering Components replicated from lower to higher security enclaves

**Notes:**
We have 4 logical instances of the Registry, each operating in a different security domain.  The submissions are registered at the lowest possible domain and are replicated to higher security domains so that a developer working at any security level will have access to all the components that instance is cleared for.

## MDR Support for Mediation

**MDR Support for Mediation**

**Convert XML to XML**
**Report status (degree of success/failure)**

**schema**

**stylesheet**

*Legend*

**2004-2005-ATO.xsl**
**(stylesheet)**

**2005-ATO.xsd**
**(schema)**

**TransformsToXmlSchema**

**2005-2004-ATO.xsl**
**(stylesheet)**

**2004-ATO.xsd**
**(schema)**

**2005-ATO.xml**
**(sample)**

**sample**

**HasXSLT**

**TransformsToXmlSchema**

**DescribedBy**

- XML Gallery Organizes
- Components for Mediation

**HasXSLT**

_***notional***_

**Registered in DoD Metadata Registry**

**Notes:**
An XSLT can transform an XML instance document conforming to one XSD to an XML instance conforming to another XSD.  In this slide, we show the information resources (XSDs, XSLTs, and sample XMLs) that were used during the Horizontal Fusion pilot.  Notice that the artifacts used in this example provide a complete round-trip transformation, from forward.xsd to reverser.xsd and back to forward.xsd.  These 5 components are registered in the HF Demo Namespace of the DoD Metadata Registry.  

## Mediation Use Case

**Message Feed**

**Client**

**Mediation listener**

** ****NCES Mediation Service**** ****Standards-based WS-I Basic Profile compliant web service to transform USMTF message to KML**

**CMP**

**Common Message Processor APIs transform USMTF text message into XML format**

**XSLT  stylesheet used to translate from USMTF schema to KML schema is retrieved from DoD Metadata Registry**

**Transformation is applied to create KML file. The result is returned to the client or stored on the Global Information Grid**

**Network links in Google Earth access the kml files and update positional  information**

**NCES **
** ****MEDIATION**

**Mediation Use Case**

## Register structural metadata 
Develop & register web services
Develop applications

**Develop & register web services**
**Develop applications**

**NCES Enabler:****Service Oriented Architecture  Foundation**

- Application

- Register
- operations

- Register
- I/O
- Structure

- * NCES Security Service

- Warfighter, Intelligence, & Business User benefit _*indirectly*_

## Using Discovered Content

**Using Discovered Content**

- Defense Knowledge
- Online*

- Federated Search*

- Federated Search*

- Federated Search*

- Database

- Information System

- Data Collection

**This database has data you are looking for**

**“****Pull the data”**

- DoD MDR

**Pull meaning, structure, & constraints**

**With access to the data source and the associated metadata, the user/application can make informed use of the data.**

- <IFF>**F**</IFF>

**What does “F” Mean?**

**Notes:**
Once a data asset is found that meets the query results, the data asset can now be accessed directly (or through an application or web interface) to “pull the data”
Metadata that describes the data asset content meaning (semantics) and structure (schema) is registered by the COI or developer in the DOD Metadata Registry (or one of the federated metadata registries).  This metadata serves as the Rosetta Stone that allows disparate sources to be understood without engineering a pre-determined interface to the data or without having anticipated apriori that this would be a possible source of data.

## MDR Support for Taxonomies

**MDR Support for Taxonomies**

- PoliticalOrganization

- TerroristOrganization

- ForeignTerroristOrganization

- AlQaida

- MyCOI
- taxonomy

- DoD Core
- taxonomy

- <ddms>
-     :
-     <Subject>**.../MyCOI.owl#AlQaida**</Subject>
- </ddms>

- Organization

- UrCOI
- taxonomy

- TerroristGroup

- Producer View

- Group

- al-Qaeda

*equals*

*equals*

- Consumer View

**Notes:**
Since we know that producers and consumers will not always share a common vocabulary, we’ve developed a strategy for filling in the Subject slot.  We’ve selected OWL as the syntax for taxonomies and their cross-references.  We’ve also added a Taxonomy Gallery to the Metadata Registry for COIs to register their taxonomies.  The Taxonomy Gallery will provide the plumbing for search processes to match consumer requests to producer offerings.  Using this strategy, we hope consumers will able to find information resources that were published using terms from a different taxonomy.  

## www.disa.mil

**www.disa.mil**

## NCES Enabler:Content Discovery

**NCES Enabler:****Content Discovery**

- Defense Knowledge
- Online*

- Other Portal
- (e.g. Intellink-S)

- Federated Search*

- Federated Search*

- Federated Search*

- Search Web Service enables federated content searches

- Database

- Information System

- Data Collection

- Application
- (machine-to-machine)

- DoD MDR

- Service
- Registry

- Supporting roles

- * NCES Security Service

- Defense Knowledge Online is _*one way*_ to use Content Discovery

**Notes:**
The search web service specification is built on the DOD Discovery Metadata Specification (from DOD net-centric data strategy).  Any data asset that is “tagged” with DDMS-compliant metadata and has implemented the search web service can be found.
In effect the search web service acts as a search of a metadata catalog of all of an organizations data holdings.  For example, the AF might crawl all of their data assets and index them with DDMS-compliant metadata (effectively creating a metadata catalog).   When a user enters a search query in any search capability that is implemented iaw the search web service (e.g, NCES portal), all AF assets that meet the search criteria will be identified.
Automated machine searching of content is achievable through the content discovery service
In response to a search query, the user/application gets back a “location” of any assets that meet the criteria so the user/application can then go directly to the source.

## Making Data Visible (1 of 2)

**DoD Discovery Metadata Specification (DDMS)**

**DDMS endorsed by **
**Executive Order 13388 **

- “Further Strengthening The Sharing Of Terrorism Information To Protect Americans”

- Data Catalog
- (historical)

**DDMS: Leverages Industry Standard**

- *

- *

- *

- *

- *

- *

- * mandatory

**Notes:**
U=Military Science

## Approach
Publish resource metadata (IAW DDMS)
Query & federate via NCES Discovery Services
Applies to...
Documents – Publish resource metadata (DDMS) in Data Catalog(s)
Services
Register web service operations in NCES Service Registry
Register I/O structures in DoD Metadata Registry

**Approach**
**Publish resource metadata (IAW DDMS)**
**Query & federate via NCES Discovery Services**
**Applies to...**
**Documents – Publish resource metadata (DDMS) in Data Catalog(s)**
**Services**
**Register web service operations in NCES Service Registry**
**Register I/O structures in DoD Metadata Registry**

- Sharepoint

**Make Data Visible (2 of 2)**

- COI
- Data
- Catalog

- Enterprise
-  Search

- Operational
- DB

- Content
- Mgmt
- Solution

- Federated
-  Search

- Federated
-  Search

- Federated
-  Search

- Federated
-  Search

- DDMS

- Unanticipated
- consumer

## Leverage Registered Domain Sets by Linking To Core

**Leverage Registered Domain Sets ****by Linking To Core**

- GMI
- FPA
- (38)

- GMI
- INCIDENT
- (241)

- GMI
- Political Subdivision
- (3784)

- MSG
- Aircraft Type
- (2889)

- GOP
- Ship Type
- (1339)

- MSG
- Ordnance Type
- (752)

- INT
- Mission Type
- (476)

- DoD Core
- Taxonomy