# Presentation: 011901

## JDO & ODBMS

- Relevant Technologies
- for Metadata Management

## Agenda

- Putting things in context
- A quick look at XMDR and ISO/IEC 11179-3 from a combined application + persistence perspective
- Focus on JDO
- What it is and why it’s important
- Thoughts about Metadata Persistence
- ODBMS is different
- Why that’s important for metadata management
- Conclusion / Summary

## ISO/IEC 11179-3: Metamodel Constructs & Terms

- Association
- Association Class
- Attribute
- Class
- Composite Attribute
- Composite Datatype
- Generalization
- Identifier (unique)
- Relationship

- Binding
- Characteristic
- Conceptual (aka Abstract) Data Model
- Data Model
- Entity
- Object
- ...

**These are all concepts that have **_**direct**_** applicability in the “object” world**

## ISO/IEC 11179-3: Some Metamodels

- Object-oriented
- Inheritance
- Lots of associations
- Data complexity!

## ISO/IEC 11179-3: And Still More!

## Metadata Content May Include ...

- Data Element Characterization
- definitions - natural language, logic-based
- types
- dimensionality / measurement units
- Taxonomies (is-a relationships) - linguistic, biological, ...
- Partonomies (part-of relationships) - geographic, organizational, anatomic, manufactured, ...
- Code Sets - e.g., country codes, CAS numbers, airport codes
- Terminologies - controlled vocabularies
- Schemas - e.g., for databases, messages, file formats, etc.
- Ontologies
- Matchings, Mappings - across terminologies, schemas, ...

## From a “Graph-Theoretic” Perspective ...

- Tree
- e.g., Dewey Decimal Classification
- an acyclic undirected graph
- Faceted Classification
- e.g., LOINC, ...
- Directed Acyclic Graph (DAG)
- e.g., instance-of relations
- also known as a acyclic digraph
- Partial Order
- e.g., is-a, and part-of relations
- Partial order = DAG + transitivity
- Lattice
- temporal intervals, sets
- General Graph
- e.g., UMLS - may contain cycles

**This all falls under the umbrella of “data complexity”**

## XMDR: Potential Approaches for Prototype

- “Traditional”
- Application Layer + Persistence Layer
- Concerns
- Application data model must be tightly coupled to the persistence abstraction
- Rigid
- John’s thoughts
- Not necessarily as bad as it sounds
- Depends on data complexity, selected repository, etc.
- In SOA, persistence issue still exists within a “service” (or at least some of them)
- DBMS-centric
- Logic moved to DBMS side of things
- Rigid, not standard, potential to lock to a specific platform
- John’s thoughts
- I agree ... Just say No!

## XMDR: Potential Approaches for Prototype (cont)

- Model Driven (Architecture -- MDA)
- Deployment code and persistence layer gets generated
- Technologies still immature
- John’s thoughts
- OMG standard ... isn’t really
- Not to be confused with “model-driven vs. data-driven” development
- Cool idea, don’t bet the farm just yet
- Fully Modular
- Plug-in based
- Flexible, clean separation of concerns
- Seems to be the preferred approach
- John’s thoughts
- Great from a SOA point of view
- Flexibility  Performance & Scalability are often mutually exclusive
- Where’s the persistence?
- Modularity doesn’t make the need go away
- View a service – or combination of services – as a “traditional” implementation

## Focus on JDO

## Java Data Objects (JDO)

- Standard for transparent Java object persistence
- Provides developers with a Java-centric and object view of persistence and data store access
- Designed to allow pluggable vendor “drivers” for accessing _any_ database/data store
- Relational, Object Database, In-memory, Flat file
- Designed to work in conjunction with J2EE
- “Connector Architecture” used to specify the contract between JDO Vendor and Application Server for instance, connection, and transaction management

## JDO from an Application Point-of-View

***JDO from an Application Point-of-View***

**Application**

**JDO**

**Transparent Persistence ...**

## JDO Abstracts Away the Disk

- The beauty of JDO is that it abstracts away the disk
- Common API for the application regardless of the storage medium
- Transparent persistence doesn’t infiltrate the developer’s code
- Allows disk-related decisions to be postponed
- Until much later in the development cycle, or
- Even until deployment!
- But under the covers, the disk is _still there_!
- The disk can’t be ignored forever
- Data must be read, written, and transferred
- The *right* storage decision is based on a number of factors
- Green field or preexisting data?
- Data complexity
- Performance needs

## Disk Oriented Point-of-View

**Application**

**JDO**

**Transparent Persistence ...**

**From the Application’s**
**Point-of-View!**

**But ...**

***Fact: Something (or someone!) must map the data model as it exists on disk into the application’s data model and vice versa!***

- It's magic!

## Why the Mapping Layer Matters

- For most repositories, mapping code must be created
- May be created by hand or via a tool
- Tools work for relatively simple / normalized data
- More direct mapping (i.e. classes to tables)
- Beyond the “simple,” manual intervention is required
- Pre-existing data may not be normalized enough for automatic mapping
- Generated JDBC or SQL may not be fast enough
- Data may be too complex

## JDO Gives Flexibility

- Decisions can be made once the facts are _truly_ known
- Decisions are non-binding!
- An error in judgment can be corrected without impacting the application
- Mapping code is required for RDBMS, not for Versant
- Even if RDBMS is necessary for deployment due to legacy application requirements, Versant’s ease of use benefits may make it a good choice for early stage development and prototyping.
- JDO makes this possible

***Fact: JDO allows application developers to delay repository decisions until much later in the development process than ever before possible***

## Benefits of JDO

- It’s a standard
- Simplifies application development
- Eliminates hand coded mapping (perhaps 40% of code)
- Don’t need to know SQL, intricacies of JDBC
- Simplified code/build/test cycles
- Greater degree of portability
- JDO vendor, database vendor, deployment architecture
- Closest thing to transparent persistence “nirvana”
- Promotes model-driven application development

- Keeps Options Open

## Thoughts about Persistence

## Application Without Persistence

- In memory only
- Data disappears at the end of program execution or failure
- Significant effort to address scalability, concurrency, etc.

**In-Memory**

-    Polygon poly = new Polygon();
-    for(i=0; i<100; i++)
-    {
-       Point p = new Point();
-       p.x = 10;
-       p.y = 20;
-       poly.contents[i] = p;
-    }
-    poly.contents[5].x = 100;

## Adding Persistence to an App

***Adding Persistence to an App***

**Disk**

**In-Memory**

- Conversion code includes
- Persistence API (e.g. ODBC, JDBC, JDO, proprietary (TopLink, CocoBase, etc.))
- Includes the generic ability to store, retrieve data
- Transactional capabilities
- Physical mapping layer
- Maps “physical” data model on disk to the application’s “logical” data model in memory

## Application Persistence

***Application Persistence***

**Application**

**Proprietary**

**JDO**

**ODBC**

**JDBC**

**Persistence**
**API**

## RDBMS Mapping Layer Challenges

**Data Complexity**

**Required Performance**

**Mapping Code**
**Implementation Difficulty**

***Fact: The farther out you get, the more likely ***
***it is that manual intervention in the ***
***mapping process will be required.***

***Fact: The more complex the data model is or the higher the performance requirements for the application, the more difficult it is to implement a mapping layer that performs and scales as required.***

## File-Based Persistence

***File-Based Persistence***

- Different in-memory and storage models
- Conversion / mapping code is required
- Implemented by hand or with a tool
- How do you manage concurrency, scalability, etc.?

**File System**

- {  convert_file_to_memory();  }
-    Polygon poly = new Polygon();
-    for(i=0; i<100; i++)
-    {
-       Point p = new Point();
-       p.x = 10;
-       p.y = 20;
-       poly.contents[i] = p;
-    }
-    poly.contents[5].x = 100;
- {  convert_memory_to_file();  }

**In-Memory**

## Using an RDBMS

***Using an RDBMS***

- Different in-memory and storage models
- Even more conversion / mapping code is required
- Separate programming and data manipulation languages
- Relies on JDBC / SQL for the database side of things
- Implemented by hand or with a tool
- Significantly degraded performance when (de)composing data

- SELECT FROM table1 ...;
- SELECT FROM table2 ...;
- convert_tables_to_memory();
-    Polygon poly = new Polygon();
-    for(i=0; i<100; i++)
-    {
-       Point p = new Point();
-       p.x = 10;
-       p.y = 20;
-       poly.contents[i] = p;
-    }
-    poly.contents[5].x = 100;
- convert_memory_to_tables();
- INSERT INTO table1 ...;
- UPDATE table2 ...;

**In-Memory**

**RDBMS**

## Versant ODBMS Solution

- In-memory and storage models are the same
- Java is both DML and DDL
- Orders of magnitude faster
- No additional conversion code
- Improved programmer productivity

**VERSANT**

-    Polygon poly = new Polygon();
-    for(i=0; i<100; i++)
-    {
-       Point p = new Point();
-       p.x = 10;
-       p.y = 20;
-       poly.contents[i] = p;
-    }
-    poly.contents[5].x = 100;

**In-Memory**

## ODBMS Supports Direct Object Navigation

- The underlying data store really does matter
- Use of tools does not create them all equal
- Relational Databases
- Object-to-relational mapping and _navigational_ joins impact performance
- Versant
- Cold navigation is faster than relational joins ... period!
- No query optimization required
- No join processing
- Anything can be related to anything ... try that with RDBMS!

## Why Use Versant?

- Reduced development time
- Cleaner, more natural design
- Application data model is the database data model
- No mapping layer
- Less code to write and maintain
- Performance
- Navigation is faster, more intuitive than joins
- Benchmarks prove this point easily
- Flexibility
- More natural design reduces impact of future change
- Lowest TCO
- Less code, more natural design  lower development cost
- Reduced need for training, administration, and management

## Conclusions

- Metadata
- By definition object-oriented
- By definition “complex”
- Regardless of architecture, it needs to be persisted
- JDO
- Given the requirements, Java appears to be a very appropriate language for development
- JDO is not intrusive into the domain model
- Appropriate for _any_ deployment architecture (2-tier, 3-tier, etc.)
- JDO “abstracts away” the disk
- Data storage technology and vendor neutral
- ODBMS
- The best choice for complex data management
- Ease of use makes it great for iterative design, prototyping, etc.
- The more complex the data, the more it will outperform RDBMS

## Reserve Info

## Proposed Prototype Architecture