# Presentation: 533431

## S.R.F.E.R.S.State, Regional, and Federal Enterprise Retrieval System

- Inter-Agency & Inter-State Integration Using GJXML

## Background

- Project Goal:
- The S.R.F.E.R.S. Project is tasked with using existing infrastructure and GJXML to build a system for sharing regional data.

## Background

- Why do we care about regional data sharing?
- 80% of justice data at local level
- Largely invisible
- Needed by:
- Neighboring Regions
- National Law Enforcement
-  Intelligence

**Notes:**
Don’t think anyone knows for sure, but the estimate I’ve heard is that 80% of justice data is locked away at the local level.

Good example of a need for regional data sharing: Phoenix and San Diego turn out to have a large population that migrates between the two cities. 

## Background

- Why aren’t these agencies sharing now?
- In most cases there are good relations.
- In most cases all parties recognize the usefulness of being able to share data.
- So what’s the problem?

## Background

- Why aren’t these agencies sharing now?
- No readily available network
- Differing record formats and data columns
- Incompatible communication systems(MQ vs. Web Services)
- Difficulty of getting data out of legacy systems.

**Notes:**
Network: Justice agencies tend to have closed networks for security reasons. It’s often difficult to gain access to a trusted network that will bridge the gap between the two local networks.

Record formats: differing formats, differing columns, etc.

Incompatible communication systems: Agencies with more mature IT departments that rely heavily on MQ Series for connecting legacy mainframe applications. On the other hand, agencies with younger IT departments tend to use Windows based computer environments and Web Services.

Legacy systems were often designed with any API. Screen scraping of mainframe applications is often the only option.

## Background

- So what are we doing about it?
- A Reusable Toolkit
- Our goal is to create a set of reusable _tools_, _methodologies_ and _infrastructure_ to solve these problems.

## Background

- No readily available network
- NLETS
- Available in every state
- Working to increase local agencies access to it.

## Background

- Differing record formats and data columns
- GJXML

## Background

- Incompatible communication systems(MQ vs. Web Services)
- Message Broker (current system)
- Centrally located in NLETS
- Translates between MQ Series and Web Services
- Rout one client request to multiple data sources. Possibly to different types of data sources.

**Notes:**
In a moment Bob will be talking about a possible new direction for SRFERS, using a system that leverages peer-to-peer technology to alleviate the problems introduced by the sheer number of local agencies potentially involved in future sharing.

## Background

- We will provide:

- A methodology
- Working examples
- A Sample Data Sharing MOU
- Privacy Impact Assessment

- Difficulty of getting data out of legacy systems.
- No silver bullet. Each agency will need to do the work to get data from their native environment into GJXML.

## Flexible Point to Point Data Sharing

**Notes:**
Most data sharing starts small with just two organizations wanting to share data. 
Our infrastructure makes it easy to build a point to point sharing application. 

In this communication model NLETS acts as an ISP that provides a network for the data to pass through.

## NLETS Message Broker

- NLETS Message Broker

**Notes:**
Over time additional agencies will join the sharing effort. Point to point queries would result in every requestor having to issue a separate query to each data provider, which would be difficult to manage. In these situations the NLETS Message Broker is used to make requests instead.

NLETS will keep a list of the providers of a given type of data. Agencies will then be able to make the data request directly to NLETS, who will in turn broadcast the request to each data provider and route the results back to the requestor. When a new agency starts providing data, the other agencies using the NLETS Message Broker will automatically start getting the new data as well.

The NLETS Message Broker system can also be expanded into a global search tool. From a single user request the Message Broker can return data from a variety of data sources at numerous locations.

## Future Directions for SRFERS

- Based on the work done in SRFERS, California will soon be providing Drivers License and Booking photos.
- Smart Search

**Notes:**
(Bob’s Presentation)

## Knowledge Exchange

- Where GJXML fits into the *S.R.F.E.R.S.* project.
- How it fits our bigger picture of inter agency integration
- What other tools and techniques were needed

**Notes:**
I have tried to cover the issues and design decisions I think are likely to be encountered by others.

There are numerous issues specific to this project that I’m skipping or glossing over. If you have specific questions feel free to ask or contact me.

## Knowledge Exchange

- GJXML is a HUGE step forward for justice data sharing. Given the large and diverse user base that it serves, creating a common “language” that we can all communicate in is a GREAT achievement.
- It is not, however, a complete integration solution. It is a tool to assist in our integration efforts.

## Topics

- What to share?
- Integration Documents
-  GJXML – what it does and does not provide.
-  An XML Wrapper

- How to share it?
- Application Structure
-  Supporting a Variety of Document Formats
-  Multi-Tier Architecture
-  NLETS Message Broker

## GJXML What it provides

- The Data Model is a nationally recognized “vocabulary” of terms for describing justice data.

## GJXML What it does not provide

- Context – GJXML represents raw justice documents, but not to do with the data.
- What to do with the record?
- Add
- Delete
- Search

## GJXML What it does not provide

- Security Information:
- Where did the record come from:
- Agency or Application ID
- Password or credential to verify the source
- Audit information
- User ID of the officer entering the report

**Notes:**
Distributed Trust Model:
By the time a request is submitted or data is sent the user has already had to log into one of their agency’s applications. We authenticate the agency. 

The user id is sent only for auditing. We assume the agency has already authenticated the user.

## GJXML What it does not provide

- Network Routing information:
- Destination address
- How to respond
- MQ Series Queue
- Correlation ID

## Topics

- What to share?
- Integration Document
-  GJXML – what it does and does not provide.
-  An XML Wrapper

- How to share it?
- Application Structure
-  Supporting a Variety of Document Formats
-  Multi-Tier Architecture
-  NLETS Message Broker

## XML Wrapper 

- There are a number of ways to convey these additional pieces of data.
- For instance if you are communicating using web services then the _**action**_ to be performed and the _**destination address**_ are implicit in the URL of the web service being called.

## XML Wrapper 

- SOLUTION: We chose to add this additional information as part of the document being passed.
- Two primary reasons:
- To be compatible with “store and forward” systems like MQ Series
- Conform to audit and logging requirements of some of the intermediate networks

## XML Wrapper 

- <xml_wrapper>
- 		<header>
*	**	**	**<Routing Information>*
*	**	**	**<Security Information>*
*	**	**	**<Context / Action>*
*	**	**	**<Response Information>*
- 		</header>
- 		<payload>
*	**	**	**<GJXML Document>*
- 		</payload>
- </xml_wrapper>

## Topics

- What to share?
- Integration Document
-  GJXML – what it does and does not provide.
-  An XML Wrapper

- How to share it?
- Application Structure
-  Supporting a Variety of Document Formats
-  Multi-Tier Architecture
-  NLETS Message Broker

## Supporting a Variety of Document Formats

- Plan to support a variety of document formats.
- These could be:
- Legacy formats
- Future versions of GJXML
- Variations in GJXML 3.0 documents from different sources

## Supporting a Variety of Document Formats

- Variations in GJXML 3.0 documents?  What!!!
- Yes, for the near future expect to see variations in the structure of the GJXML you receive from 3rd parties:
- Within the GJXML system there are often a number of ways to create the same document.
- IEPs are being designed, but they overlap and in many cases there is no clear adopted ‘standard’ IEP for a given document type. There are also MANY application specific schemas used by vendor products.

**Notes:**
This is not a knock on GJXML. It’s an evolving standard and the reality is there will be various interpretations of it at first.

## Supporting a Variety of Document Formats

- Don't stove pipe.
- Handling each document type separately is termed “stove pipe” integration.
- It the simplest approach, but is hard to maintain and upgrade:
- Business rules must be copied for each
- Data validation must be done for each
- Often leads to a hard link between two applications instead of a reusable interface

## Supporting a Variety of Document Formats

- SOLUTION: Convert all 3rd party documents into your own document format.

**Notes:**
Reuse, reuse, reuse!!!!

## Supporting a Variety of Document Formats

- Your native format could be:
- Your version of GJXML which contains all the fields your system understands.
- A legacy format you are already setup to use.
- If you’re a Java or .Net shop it may be a native Data Object which has variables for each of your data fields.

## Topics

- What to share?
- Integration Document
-  GJXML – what it does and does not provide.
-  An XML Wrapper

- How to share it?
- Application Structure
-  Supporting a Variety of Document Formats
-  Multi-Tier Architecture
-  NLETS Message Broker

## Multi-Tier Architecture

- A multi-tier application architecture is a good practice in an enterprise environment.
- Promotes code reuse
- Simplifies maintenance
- Enhances system flexibility
- e.g. Multiple documents types for the same functionality

## Multi-Tier Architecture

- The architecture we decided on within ARJIS.

**Notes:**
Our Presentation Tier is the external interfaces.

The standard layers are:
Presentation Tier – Logic related to displaying data
Middle Tier – Business logic of the application
Database Tier

Native Interfaces and Database API could be web services, but are more likely a programmatic interface, such as an EJB or .NET object. They may use a queue (JMS or MQ Series) to receive data. 

## Topics

- What to share?
- Integration Document
-  GJXML – what it does and does not provide.
-  An XML Wrapper

- How to share it?
- Application Structure
-  Supporting a Variety of Document Formats
-  Multi-Tier Architecture
-  NLETS Message Broker

## NLETS Network Broker

- We had 2 design goals for interoperability between agencies.
- Wanted each client to be able to communicate without needing to know about the recipient.
- Specifically we need to support both:
- Web Services
- MQ Series

## NLETS Network Broker

- 2) Retrieve data from multiple sources with a single request.
- Nice To Have:As new data sources come online we wanted a way to easily make them available to the agencies already using the system.

## NLETS Network Broker

- SOLUTION: NLETS has setting up a “broker” that handle these network needs.
- Receives both MQ or Web Service messages.
- Sends on either MQ or Web Service messages, depending on the destination.
- Routs responses back using the correct protocols
- Has the ability to broadcast a message to multiple destinations.

**Notes:**
Since requests and responses are pre-defined, new agencies will be able to easily plug into the system. Their data responses will just start appearing in the result sets of existing agencies.

## NOOGLESearch

- NOOGLESearch

**Notes:**
The NLETS Message Broker system can also be expanded into a global search tool. From a single user request the Message Broker can return data from a variety of data sources at numerous locations.

## Topics

- What to share?
- Integration Document
-  GJXML – what it does and does not provide.
-  An XML Wrapper

- How to share it?
- Application Structure
-  Supporting a Variety of Document Formats
-  Multi-Tier Architecture
-  NLETS Message Broker

## Contact Info.

- Dustin Henson
- ARJIS San Diego
- (Automated Regional Justice Information System)
- _dhenson@sandiego.gov_
- Bob Slaski
- NLETS
- The International Justice and Public Safety Information Sharing Network
- _bslaski@nlets.org_