# Presentation: 355066

## XMDR Prototype Progress Report

**XMDR Prototype Progress Report**

**John McCarthy and Karlo Berket**
**XMDR Project Meeting**
**January 23, 2007**
**Faculty Club**
**University of California**
**Berkeley**

**Notes:**
Karlo Berket joined LBL’s XMDR Project group as a part time programmer in January 2006
He comes to us from the Cybersecurity and Network :& Distributed Systems Research Group at LBNL,
which continues to occupy and pay for the other half of his working hours

## XML Prototype Progress Outline

**Semi-automated UML to OWL translation of ISO/IEC11179 working draft 4**
**Revised architecture & module diagrams**
**Content (re)loaded to date & planned**
**Coping with scale (Omega ontology, continued)**
**Demonstrate revised XMDR Prototype (2007.01.19) **
**Integrated text and inference queries & results **
**XMDR portal -- software, data & documentation**
**Next steps & major challenges**

**Notes:**
Prepared this with Larry in mind as the architype for primary audience

Tell story about painful upgrade from T23 to T43
Analogy to upgrading 11179 to include more formal semantic information
We’ve been hearing about inference from Frank for a number of years (Prolog, etc.)

Problem that had a silver lining
Karlo machine crashed, so he had to re-install—did it on xmdr server and revised recipe

## 11179-ed2 used UML to specify metamodel, but only via human database designers

**ISO/IEC 11179 ed3**
**Metamodel (in UML)**

**11179 **
**Relational **
**Schema**

**Relational **
**Database**

- human translation

**Registry Content**

**Notes:**
In the future, XML schema may be able to enforce most of the constraints from OWL, RDF, and XML automatically

Can create regular XML Schema from Relax-NG specs using TRANG

## XSLT scripts used to generate OWL from ISO11179 WD4 UML specs

**Needed quicker prototype updates from 11179 draft specs**
**Current automation tools did not work**
**tools use UML2, but 11179 spec was in UML1.x**
**but even UML 2 from Poseidon ****3.2.1-0**** did not work**
**unsuccessful loading UML2 XMDR into TopBraid & Sandpiper**
**UML2 not as yet as interchangable as hoped (more from Elisa on Thurs)**
**Created XSLT script(s) for converting UML to OWL**
**Current version of scripts do not (yet)...**
**Separate packages into separate namespaces**
**Create owl:disjointWith properties**
**Translate OCL rules/restrictions**
**(e.g., registered item is either an administered item or an attached item)**

**Notes:**
UML 2 from Poseidon may have failed either due to Poseidon or to non-compliant XMDR specs

## XMI from 11179-ed3 UML is transformed to RDF/OWL for XML metamodel specification

**XMI from 11179-ed3 UML is transformed to RDF/OWL for XML metamodel specification**

**XMDR OPEN **
**SOURCE TOOLS**

**FILES**
**(AND TYPES)**

**POSSIBLE**
**ALTERNATE**
**TOOLS**

**XMDR**
**Metamodel**
**(UML)**

**XMDR**
**Metamodel**
**(XMI)**

**XMDR**
**Metamodel**
**(preliminary**
**RDF/OWL)**

**XMDR**
**Metamodel**
**(final**
**RDF/OWL)**

**XMDR**
**Metamodel**
**inferred**
**logic index**

**UML modeling**
** ****& editing**

**ABSTRACT**
**STEP**

**UML interchange**

**Transformation **
**to OWL**

**OWL ontology**
**display,** **editing **
**& validation**

**Logic index**

**Reasoner**

**Human compare / hand editing**

**XMDR**
**Metamodel**
**asserted**
**logic index**

**Poseidon**

**XSLT**
**scripts**

**Protege**

**Swoop**

**Jena**

**Pellet**

## OLD XMDR Prototype Architecture Diagram & Implemented Modules

**Ontology Editor**

**Protege**

**11179 OWL Ontology**

**OLD XMDR Prototype Architecture Diagram & Implemented Modules**

**Authentication Service (defer)**

**MappingEngine (defer)**

**Registry**

**External**
**Interface**

**MetadataValidator**

**XML Schema (for XML)**
**Jena (for RDF)**
**Protégé & Swoop (for OWL)**

**Java**

**RetrievalIndex**

**FullTextIndex**

**Lucene**

**LogicBasedIndex**

**Jena, [Sesame?]**

**RegistryStore**

**WritableRegistryStore**

**Subversion**

**Notes:**
Tools in smaller different font &
Lines around boxes for print version
Black print on pastel blocks

Show Protégé creating ontology used to define storage

Evaluated OWI KS and Racer but they had performance and other issues (see data in prior presentation)

Paid version of Saxon may be able to do schema-driven transformations

Karlo says mapping between concept systems could be done by loading additional OWL files
Mapping happens at 2 levels: between different instances of same OWL class; and between dif instances within a class (e.g., Data Elements)

## XMDR Prototype Modular Architecture

**XMDR Prototype Modular Architecture**

**Registry Store**** ****(Subversion)**

**Search & Inference Queries**

**XMDR metamodel **
**(OWL & xml schema)**

**Full Text**
**Index**

**standard XMDR files**

**standard XMDR files**

**standard XMDR files**

**standard XMDR files**

**Asserted**
**LogicIndex**

**Inferred**
**LogicIndex**

**Content Loading & **
**Transformation**

**Human User Interface**

**Metadata Sources **
**concept systems,**
**data elements**

**USERS**
**Web Browsers.....Client**
**Software**

**Application Program Interface**

**Authentication Service**

**Validation**
**(XML Schema)**

**Mapping**
**Engine**

**Reasoner**

**Text Search**

**Metamodel specs**
**(UML & Editing)**

**XMDR data model **
**& exchange format**
**XML, RDF, OWL**

## XMDR Prototype Modular Architecture:with current open source software selections

**XMDR Prototype Modular Architecture:****with current open source software selections**

**Registry Store**** ****(Subversion)**

**Search & Inference Queries**** (Jena, SPARQL)**

**XMDR metamodel **
**(OWL & xml schema)**

**Full Text**
**Index**

**XMDR Prototype Architecture    REST Style**

**standard XMDR files**

**standard XMDR files**

**standard XMDR files**

**standard XMDR files**

**Asserted**
**LogicIndex**

**Inferred**
**LogicIndex**

**Content Loading & **
**Transformation**
**(Lexgrid & custom)**

**Human User Interface**
**(XML pages & javascript)**

**Metadata Sources **
**concept systems,**
**data elements**

**USERS**
**Web Browsers.....Client**
**Software**

**Application Program Interface**** ****(REST)**

**Authentication Service**

**Validation**
**(XML Schema)**

**Mapping**
**Engine**

**Reasoner**
**(Pellet)**

**Text Search**
**(Lucene)**

**Metamodel specs**
**(UML & Editing)**
**(Poseidon, Protege)**

**XMDR data model **
**& exchange format**
**XML, RDF, OWL**

## Criteria for XMDR Prototype software selctions

**Open Source (vs. commercial)**
**Functionality**
**Performance**
**Scalability**
**Modularity (ability to combine with other software)**
**Cost**
**Availability**
**Operating System dependencies**

**Notes:**
Perhaps a third slide could suggest some criteria for software selections: Open Source vs. commercial, functionality, performance, scalability, cost, maintenance, availability, Operating System dependencies, ....

## Content loading, with XMDR metamodel used for inferred indexing and validation

**Content loading, with XMDR metamodel used for inferred indexing and validation**

**CONTENT**

**Terminology A**

**Terminology D**

**Terminology B**

**Thesaurus C**

**Data Element**
**Source E**

**Terminology**
**Source F**

**Ontology**
**Source G**

**External**
**Source H**

**VALIDATION**

**TRANSFORMATION**

**REGISTRY INFORMATION**

**INDEXING**

**Lexgrid**

**Reasoner**
**(Pellet)**

**Text**
**Indexing**
**(Lucene)**

**Inferred**
**LogicIndex**

**Assesed**
**LogicIndex**

**Full Text**
**LogicIndex**

**Search &**
**Inference**
**Framework**
**(Jena)**

**XSLT**
**script E**

**XSLT**
**script F**

**XSLT**
**script G**

**XSLT**
**script G**

**XMDR Files A**

**XMDR Files D**

**XMDR Files B**

**XMDR Files C**

**XMDR Files E**

**XMDR Files F**

**XMDR Files G**

**XMDR Files H (virtual)**

**Subversion**

**XMDR **
**metamodel**
**In XML **
**schema**

- From OWL

**Notes:**
NEED to explain what Lexgrid is

## Example concept system content currently loaded into XMDR Prototype

***via Lexgrid***
**National Biological Information Infrastructure biodiversity **
**NCI Thesaurus_06.02d health concepts system**
**GEMET 2001.0 Multilingual Environmental Thesaurus **
**ISO4217_1981 currency codes**
**ISO3166_V-10 country codes (only 2 letter codes)**
**Mouse_1.32  anatomy**
**Defense Technology Information Center 1.0 Thesaurus**
**Portions of EPA controlled vocabulary**
**SIC and NAICS industrial classification codes**
***via special purpose scripts***
**Omega ontology**

**Notes:**
Show relative size of each

## Additional Metadata Content planned for XMDR Prototype

**Current 11179 Data Element Registries**
**caDSR (full NCI Cancer Data Standards Registry)**
**EDR (EPA Environmental Data Registry)**
**Candidate Additions to Concept Systems and Ontologies**
**NASA SWEET (Semantic Web Earth & Environmental Terminologies)**
**IETF RFC 3066 Language Codes**
**USGS Geographic Names Information System**
**Getty Thesaurus of Geographic Names**
**I.T.I.S. - Integrated Taxonomic Information System**
**Foundational Model of Anatomy **
**EPA Chemical Substance Registry **
**GO (Gene Ontology), ....Agrovoc,  ...and possibly others**

- 2005-07-27

**Notes:**
Main criteria are 
   wide range of substantive fields of interest
   different types of data structures, complexity, and other types of challenges

## Omega Ontology illustrates challenges of loading large, complex new content

**Omega is a “terminological ontology” **
**reorganization & synthesis of WordNet & Mikrokosmos**
**adds higher level ontology to organize multiple ontologies**
**Initial mapping and loading of Omega needs to be refined**
**Entity relationships conform to Concept_System figure **
**Entity ->Attribute conforms to Classification_Scheme figure**
**Omega Attributes mapped to ISO/IEC11179 ed3 Facets**
**(ignoring Omega datatype field)**
**Required a week to process and load Omega Ontology **
**4 million files, so ~250,000/24 hrs**

**Notes:**
Frank says someone at ICSI working on combining WordNet with upper level ontology (maybe SUMO)

## NCI Thesaurus is also challenging

**148,110 files of input data **
**One per each concept, link, relation and relation role**** **
**Number of asserted files are 14x larger  (2,149,974) **
**Number of inferred files are 35x larger (5,305,344)**
**Size increases 1 to 2 orders of magnitude**

**Notes:**
Ask Karlo about 148,110 files of input data (one per concept?)

## XMDR Prototype approach has evolved to overcome semantic web limitations

**Semantic web premised on getting lots of pieces and then inferring from them – not retrieving from large databases**
**Reasoners require that working set is in memory**
**XMDR content differs from this paradigm (lots of content)**
**Inference at run-time infeasible**
**Inference across all content infeasible**
**Latest XMDR software performs reasoning at load time**
**Have to re-load all data if rules change**
**But no more memory issues at run-time**
**Analogous to materialized views in relational database technology**
**Latest XMDR software does reasoning on parts of content**
**Portions 40K-100K asserted triples**
**Limits reasoning**

## Future Possibilities

**Commercial triple stores claim to scale better**
**allegrograph/racer**
**Ontology Works/Objectivity)**
**Reasoners that don’t need everything in memory?**
**Reasoners that can work with incremental changes (next version of Pellet will support this)**
**Assert everything in generating XMDR content -- leave mapping module to deal with reasoning**

## Current coverage of 11179 classes in XMDR prototype system

**Concept Systems *****e.g., NBII, NCI Thesaurus***
**Classification Schemes *****e.g., CDISC Codelists***
**Conceptual Domains *****e.g., Countries of the World***
**Characteristics *****e.g., Examined, Analyzed ***
**Object Classes *****e.g., Participant, Finding ***
**Data Element Concepts *****e.g., Country Label ***
**Data Elements *****e.g., Country Name ***
**Value Domains *****e.g., countries of the world***

**Concepts *****e.g., River outflow***
**Relations *****e.g., IsA, PartOf, broader, Allele_Has_Activity***
**Links**
**Organizations *****e.g., EPA***
**Units of Measure *****e.g., seconds, ******ml/min,*** ...

**For number of instances of each, see**
**xmdr.lbl.gov/xmdr/coverage.jsp**

## web interface combines inference & text queries xmdr.lbl.gov/xmdr/

**web interface combines inference ****& text queries **xmdr.lbl.gov/xmdr/

- or from

- or pellet

- or pre-set number of instances (10,20,30...50)

- new feature

- resources (URIs) or literals (values)

**Notes:**
Use diff color for annotations!
Frank asks What kinds of inference? Transitive, simple boolean, more general first order logic
Do we allow user-specified predicates? Or just what is in the model?
NEED NEW INTERFACE
In the future, will be able to choose multiple items from pick lists
If you choose number of results to display, it won’t display how many are retrieved in total at the top because it starts as soon as results come back
Will keep text search because it is Lucene only – simple, ala google
Concept with text -- Same with text search  Administrered Item  Water contamination

XMDR Item Type
In general, XMDR Item Types are names of types of metadata items of interest. In particular, they are class names taken from the OWL specification of the XMDR metamodel, which is based on the ISO/IEC 11179 Part 3 Version 2 metamodel and very similar (but not always identical) to the proposed ISO/IEC 11179 Part 3 Edition 3 metamodel. 

XMDR Links
These are properties from the XMDR metamodel, i.e., names of associations and/or attributes of the classes from the metamodel (XMDR Item Types). 
The direction of the link, i.e., to or from choice, indicates whether search results should include entities which link to or from the specified target (selected attribute or relationship) 

Result Type
This specifies whether to return resources (specified by URIs -- e.g. web addresses) or literals (text strings, numbers, etc.) or both. 
The anonymous resources checkbox indicates whether or not to return information about blank (unnamed) RDF nodes as part of the search results. 

Inference Engine
What type of inference engine to use in conjunction with the queries. No reasoner means just return exact matches for the query. The various inference engines allow one to return various implied results, e.g., the result of performing a transitive closure on specified types of links. 

Limit results to
Further restricts search results on the basis of their text contents. 

## Inference query results

**Inference query results**

**URI prefixes**

**SRARQL Query**

**Number of Concept (or subclasses) Items/Files Found**

**file identifier, with prefix**

**show detailed information**

**designated item sign(s) for each item/file**

***What other summary information should we show for each item/file?***

***See relevant part of XMDR metamodel diagram***

**Notes:**
INFO link shows detailed information about the item (including inferred info -- shown on next slide)

## “Info” shows details about items with asserted info at left

**“****Info” shows details about items with asserted info at left**

***See relevant parts of metamodel diagram***

- URI – item/file ID

- Links

- non-bold shows attributes of bold items

***more on next page!***

- tags on left

- actual data rather than links

- items that are neither identified or registered

- another anonymous node

- values on right

- Pages divided into two sections –
- stored data at the left ......and inferred data on right

- Designation.Sign

**Notes:**
KARLO: REPLACE

## Inferred information in right half

**Inferred information in right half**

- Designation.Sign

- URI – item/file ID

- Links

***How can we improve the content & format ***
***of this detailed data for each item?***

***more on next page!***

- non-bold shows attributes of bold items

- Pages divided into two sections –
- stored data at the left ......and inferred data on right

**Notes:**
KARLO: REPLACE

## Info detail incoming links withinferred information on the right

**Info detail incoming links with****inferred information on the right**

- Legend of Prefixes

**Notes:**
KARLO: yes, replace (if showing new version) 

## Item detail ends with incoming links and legend of prefixes

**Item detail ends with incoming links and legend of prefixes**

- Legend of Prefixes

**Notes:**
KARLO: yes, replace (if showing new version) 

## Demo & Discuss XMDR

**List of Concept_System instances in prototype ****	**
**“****River outflow” Reference_Concept from NBII**
**“****UseFor” Relation_Role from NBII**
**Broader relations (from interface example)**
**Any other requests**

**Notes:**
KARLO: REPLACE IF SHOWING NEW VERSION

## Notable features of XMDR Advanced Inference Search

**You don’t have to know SPARQL**
**but you can see the generated SPARQL query**
**Each search component has pop-up help screen **
**Choice of reasoners **
***None, Pellet***
**Can restrict search to target object type**
***e.g., concept system, data element, concept, value domain, etc. ***
**Can restrict search by object attributes or links**
***e.g., administrativeStatus, designation, etc.***
**Combines XMDR text search (via Jena)**
***phrases, words (all, at least one, without), strings***
**Simple output summary & control**
***Result count, specify number displayed per screen***
***Show results as web addresses, literals, or both***
***Restrict search by Concept System (new)***

**Notes:**
KARLO has updated object type and attribute names to current specification 

## XMDR Prototype Web Site has downloadable code & content

**http://xmdr.lbl.gov/**

- Note tabs for other sections!

**Notes:**
KARLO: MAY NEED NEW SCREENSHOT

## New XMDR checkpoint release includes several improvements

**Uses new XMDR ontology based on iso11179 part 3 edition 3 working draft 4**
**Improved memory footprint**
**Improved mixed searches (logic & text)**
**Improved times for Advanced Inference Search**
**Core functionality now distinct from 11179 model**
**New option to constrain search to a particular concept system in Advanced Inference Search**
_**[http://](http://xmdr.lbl.gov/software/releases.html)**__**[xmdr.lbl.gov/software/releases.html](http://xmdr.lbl.gov/software/releases.html)**_

## Next priorities for XMDR Prototypeare currently under discussion

**Add more metadata  **
**fully populate all aspects of 11179 metamodel**
**especially for example 11179 data registries, i.e. caDSR, EPA-EDR**
**other content that stretches the current model (like Omega)**
**Improve tools & procedures for input data mapping/loading**
**More extensive integration of Lexgrid features**
**Extend XMDR System Features**
**try using Lexgrid for other functionality beyond data loading**
**e.g., Ontology Lifecycle Management**** – versions & semantic drift**
**experiment with other User Interface options**
**Exhibit from MIT-Sesame project**
**BioPortal**
**transitive closure queries within specified number of arcs**
**let users select multiple items from interface pick lists**
**Investigate microformats (**_**[http://microformats.org/about/](http://microformats.org/about/)**_** )**

**Notes:**
SEE NSF Milestones

## Technical Challenges and Issues for XMDR Implementation Testbed

**Complexity**
**Representation of relations**
**Omega ontology has raised a number of issues**
**how to provide extensibility for unknown future complexities?**
**Scalability & performance**
**Currently includes [**_***number***_**] objects & [**_***number***_**] RDF triples**
**maybe indexing and/or distributed registries will help?**
**Model Evolution**
**we eventually hope to generate directly from UML**
**External metadata sources, ontologies, terminologies**
**Mapping (e.g. between concept systems, to data elements)**
**Harmonize with ODM, MMF, Common Logic, Web Services **

**Notes:**
NEED NUMBERS ABOVE

## Thanks & Acknowledgements

**Bruce Bargmeyer, Principal Investigator**
**Frank Olken, Theory & Model Development**
**Harold Solbrig, Lexgrid, Model Development, etc!**
**L8 and SC 32/WG 2 Standards Committees**
**Major XMDR Project Sponsors and Collaborators**
**National Science Foundation (Grant #0637122)**
**U.S. Environmental Protection Agency**
**Department of Defense**
**National Cancer Institute**
**U.S. Geological Survey**
**And others!**

## Contact Information

**XMDR Project**
**http://xmdr.org/**
**Bruce Bargmeyer, Principle Investigator**
**BEBargmeyer@lbl.gov**
**Karlo Berket, Systems Design & Programming**
**KBerket@lbl.gov**
**John McCarthy, Information Systems Consultant**
**JLMcCarthy@lbl.gov**

## *Info shows details about items (including inferred info)

***How can we improve the content and format of this detailed data for each item?***

***See relevant parts of metamodel diagram***

***more on next page!***

- non-bold shows attributes of bold items

- tag on left with value on right

- Page divided into two sections – stored data on left & inferred data on right

- tag on left with value on right

- actual data rather than links

- items that are neither identified or registered

- another anonymous node

- NBII Concept System

- Designation.Sign

- Designation.Sign

- URI – item/file ID

- Links

**Notes:**
KARLO: REPLACE

## *Continuation of Item detail

- stored data on left & inferred data on right

- Legend of Prefixes

- nodes

- arrows

**Notes:**
KARLO: yes, replace (if showing new version) 

## DEFER XMDR Prototype example: dual purpose rdf/xml file (extract) for one GEMET term

**DEFER XMDR Prototype example: dual purpose rdf/xml file (extract) for one GEMET term**

**<Reference_Concept xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"**
**                   ****xmlns="http://hpcrd.lbl.gov/SDM/XMDR/ont/iso11179-3e3draft_r1_7.owl#"**
**                   ****xml:base="http://xmdr.lbl.gov/xmdr2/data/OMEGA-4/R-C/50010/1451.xml"**
**                   ****rdf:about="">**
**   ****<Identified_Item.data_identifier rdf:datatype="http://www.w3.org/2001/XMLSchema#string">OMEGA-4/R-C/50010/1451.xml</Identified_Item.data_identifier>**
**   ****<Identified_Item.version rdf:datatype="http://www.w3.org/2001/XMLSchema#string">4</Identified_Item.version>**
**   ****<Identified_Item.identification_source rdf:resource="http://xmdr.lbl.gov/xmdr2/data/OMEGA-4/N/5001.xml"/>**
**   ****<Designatable_Item.designation rdf:parseType="Resource">**
**      ****<Designation.sign rdf:datatype="http://www.w3.org/2001/XMLSchema#string">table tennis</Designation.sign>**
**      ****<Designation.designation_context_relevant_designation rdf:parseType="Resource">**
**         ****<Designation_Context.scope rdf:resource="http://xmdr.lbl.gov/xmdr2/data/OMEGA-4/C-1.xml"/>**
**      ****</Designation.designation_context_relevant_designation>**
**   ****</Designatable_Item.designation>**
** ****<Concept.container rdf:resource="http://xmdr.lbl.gov/xmdr2/data/OMEGA-4/CS.xml"/>**
**</Reference_Concept>**

**Karlo show new version**
**Annotate parts that **
**illustrate RDF & OWL**

- 28

**Notes:**
KARLO suggests we try showing more compact representation of triples (using N3 or similar tool)
Perhaps in addition to the full blown XMDR RDF/XML format such as above 
Is this possible now Karlo?

Concept is shorthand in xmdr name
Property is RDF type subject

Subject property object
Xml serialization 
Class name for tag is shorthand for property