# Presentation: 692501

## NLETS Lessons Learned

**NLETS Lessons Learned**

## Kate Silhol

NLETS Senior Software Engineer
ksilhol@nlets.org
(602) 224-0744
www.nlets.org

*NLETS Senior Software Engineer*
- ksilhol@nlets.org
- (602) 224-0744
- www.nlets.org

## What is NLETS?

**What is NLETS?**

**Notes:**
Its sole purpose is to provide for the international,  interstate and/or interagency exchange of criminal justice and criminal justice related information.
Our Mission:
...to provide, within a secure environment, an international criminal justice telecommunications capability

NLETS will assist those national and international governmental agencies and other organizations with similar missions

## NLETS is:

- Members are all 50 states, most federal agencies & territories, and the RCMP
- Nearly 40 years in operation
- Links 30,000 agencies and over 500,000 access devices in the U.S. and Canada
- Topping 40 million transmissions per month

**NLETS is:**

## Facts and Figures

- 41 Million transactions per month!  _*Only*_ way to:
- State Criminal Histories
- Electronic “HIT” confirmation
- Canadian Files (CPIC)
- Homeland Alert Messages
- Screen Officers Flying Armed
- INS databases at LESC – IAQs
- Driver records and vehicle registrations
- Aircraft Registrations
- HAZMAT, GSA Federal registrations, NICB
- Push administrative messages to all CJ agencies

## Services:

- Private T1 Network, NO Internet
- VPN – Triple DES encryption
- Fire-walled at every connection
- Hot Disaster Recovery Site - Idaho
- Store and Forward – Guaranteed Message Delivery
- Operation center 7 x 24 x 365 - Arizona
- 99.6% uptime

## Requirements

- Represent precise data
- Reliability
- Performance
- Flexibility
- Maintainability

## NLETS XML Operational Capabilities

- XML Message Router (XMR) since 2001
- All NLETS transactions available in GJXDM 3.0 format plus full legacysystem translation capabilities since September
- 750,000 XML rap sheets alone per month
- All FBI, WI and KY rapsheets are in XML
- All NLETS states/agencies have access to XML and web services

## XML Formats and Transport

- NLETS allows users to send XML via Web Services, MQ Series or TCP/IP
- Allowing XML over various protocols is crucial to users in our environment
- NLETS transforms between XML and legacy text to allow interoperability between users
- All new data providers will be connecting via Web Services

## Validates user

**Validates user**

**Transforms XML to Text**

- XML Message

- Text Message

**XMR**

**Determines Destination & Format**

**Validates XML**

- XML Message

- Text Message

- Text/CDATA XML Message

## Web Services

- Within a secured network, NLETS exposes a web service to which users connect and send XML as a string
- States wishing to receive a response via web services host own service for NLETS to connect to
- NLETS specifies name and fingerprint of service, how data is handled is up to state

## NLETS Standardized GJXDM XML Transactions

- All Inquiries
- Standard text format is easy to convert to/from XML
- Criminal History Responses
- Leveraging the JTF Criminal History Rapsheet Specification
- Hazardous Materials Responses
- Standardized single-source responses from Operation Respond
- Drivers and Registration Responses
- Standardized responses from state DMVs based on CANDLE grant products

## So, how did we do it?

- Mapped elements to GJXDM
- Documented XML Instances
- Created applications to consume XML
- Created schemas
- Get users to start implementing
- Change management

## Mapping NLETS Transactions

- First, created a
-      generic message
-      envelope, in NLETS
-      namespace (prefix n)
- Header information
- 	remains in the payload
- 	instead of SOAP
- Multiple transports
- Legacy header data

- <n:NLETS>
-     <n:NLETSInquiryMessage>
-         <n:NLETSInquiryHeader>
-              ..... NLETS header elements ......
-         </n:NLETSInquiryHeader>
-         <n:NLETSInquiryData>
-              ..... NLETS transaction data ......
-         </n:NLETSInquiryData>
-     </n:NLETSInquiryMessage>
- </n:NLETS>

## Mapping NLETS Transactions

- Next, within the generic NLETS message envelope we defined each transaction in XML
- Going from a legacy system provided us a strictly defined set of information
- Used GJXDM elements where available
- Extended GJXDM elements and types when possible
- Example: Agency
- Participated in GJXDM development by utilizing feedback mechanisms (Bugzilla, XSTF)

## NLETS GJXDM Example

- <n:NLETS>
-    <n:NLETSMailHeader>
- 	<n:MessageKeyCode>RQ</n:MessageKeyCode>
- 	<j:DocumentSource.Organization>
- 		<j:OrganizationORIID>
- 	   	      <j:ID>GA0250300</j:ID>
- 		</j:OrganizationORIID>
-    	</j:DocumentSource.Organization>
-        <n:DocumentDestinationID>
-                  <j:ID>AZ</j:ID>
-         </n:DocumentDestinationID>
-         <n:DocumentControlFieldText>TERM000000</n:DocumentControlFieldText>
-    </n:NLETSMailHeader>
-   <n:NLETSInquiryData Key=“RQ”>
-       <j:VehicleRegistrationPlateID>
-              <j:ID>LJB934</j:ID>
-       </j:VehicleRegistrationPlateID>
-       <j:VehicleRegistrationPlateTypeCode>PC</j:VehicleRegistrationPlateTypeCode >
-       <n:IDExpirationDateText>1983</n:IDExpirationDateText>
-   </n:NLETSInquiryData>
- </n:NLETS>

- RQ.GA0250300.AZ.*TERM000000.TXT
- LIC/LJB934.LIY/1983.LIT/PC

## NLETS GJXDM Header

- <n:NLETSMailHeader>
- 	<n:MessageKeyCode>RQ</n:MessageKeyCode>
- 	<j:DocumentSource.Organization>
- 		<j:OrganizationORIID>
- 	   	      <j:ID>GA0250300</j:ID>
- 		</j:OrganizationORIID>
-    	</j:DocumentSource.Organization>
-        <n:DocumentDestinationID>
-                  <j:ID>AZ</j:ID>
-         </n:DocumentDestinationID>
-   <n:DocumentControlFieldText>TERM000000</n:DocumentControlFieldText>
- </n:NLETSMailHeader>

- RQ.GA0250300.AZ.*TERM000000.TXT

## NLETS GJXDM Data

-  <n:NLETSInquiryData Key=“RQ”>
-       <j:VehicleRegistrationPlateID>
-              <j:ID>LJB934</j:ID>
-       </j:VehicleRegistrationPlateID>
-       <j:VehicleRegistrationPlateTypeCode>PC</j:VehicleRegistrationPlateTypeCode >
-       <n:IDExpirationDateText>1983</n:IDExpirationDateText>
- </n:NLETSInquiryData>

- LIC/LJB934.LIY/1983.LIT/PC

## Supporting legacy text values
Legacy text may have supported nonconforming data values (example: Expiration Date)
Newly standardized transactions
Cannot find structure in older, non-standardized formats
Must support two versions of XML “3.0”
Multiple ways to represent same data
GJXDM not meant to support certain types of data
Messaging/Header information

- Legacy text may have supported nonconforming data values (example: Expiration Date)
- Newly standardized transactions
- Cannot find structure in older, non-standardized formats
- Must support two versions of XML “3.0”
- Multiple ways to represent same data
- GJXDM not meant to support certain types of data
- Messaging/Header information

**GJXDM Mapping Challenges**

## Schemas

- Each message type has its own schema “package”
- Combined Document/Extension schema
- Defines overall structure of message – referencing GJXDM elements and defining NLETS-specific elements
- Global JXDM constraint schema
- Defines GJXDM elements being used and specifies constraints (min/max occurrences)
- Shared supporting schemas
- NCIC, NIBRS code enumerations, etc

## Schemas

- Validates against

- Imports

- Imports

## Schemas

- NLETS does not currently perform schema validation on each transaction
- Table-based validation of data
- Validate users’ XML in a test environment first, then trust them
- Schemas are provided primarily as documentation for programmers
- Recommend use during development and testing
- Schemas cannot communicate all rules for XML conformance

## Documentation of XML Specifications

- XML is defined in detail in Appendix A of the NLETS User Guide
- For each transaction, schemas, XML instances, stylesheets and _business rules_ are provided

## NLETS processes, switches data but does not create, store or manipulate data
Must parse text and XML in order to route, validate and convert
XPath for XML, regular expressions for text

- Must parse text and XML in order to route, validate and convert
- XPath for XML, regular expressions for text

**Consuming XML**

## Standardized Rap Sheets

- Currently Wisconsin, Kentucky, Maine and FBI sending and/or receiving rapsheets in XML – JTF 2.2 format
- New states will be sending JTF 3.0 XML rapsheets (compliant with GJXDM 3.0)
- All users receiving standard presentation format – most in legacy formats
- All III Inquires done - go through and are delivered by – NLETS

**Notes:**
Complies with GJXDM V3.0 
Interstate Criminal History Transmission Specification V3.0 
NLETS will support XML Rap Sheet V3.0 and V.2.2x but no transformations will occur between 3.0 and 2.2 - V2.2x will only be delivered as text

## Mapping the Rapsheet

- Similar to the process followed in mapping NLETS transactions
- While we built from an existing specification, we had more flexibility
- Benefited from GJXDM data dictionary
- Data we had not previously thought to include
- Provides more clear consistency throughout rapsheet
- Found a greater need to extend elements to restrict allowable values

## CANDLE

**National Institute of Justice**
**Bureau of Justice Assistance**
**NLETS (Awardee)**
**American Association of Motor Vehicle Administrators**
**Advanced Technology Systems**
**Pilot States**

**BJA**

**Collaboration between AAMVA and NLETS****for Driver License Exchange**

## CANDLE

- Standardized GJXDM XML representation of DMV drivers license and registration responses
- Unlike NLETS Inquiries and Rapsheets, CANDLE specifications were built from scratch – not existing spec
- Cannot convert text driver and registration responses to XML

## Operation Respond

**HAZMAT Information - live**
**Trucking manifests and rollover data**
**Rail Car Data**
**Passenger Train Schematics**
**Aerial photography via NLETS – a possibility**

## Operation Respond

- Not a criminal justice agency, though information is pertinent to CJ agencies
- GJXDM did not contain hazardous material elements
- Created elements in Operation Respond’s namespace, utilizing same naming conventions and GJXDM elements when available

## Change Management

- Even small changes affect entire user base
- User concern about a “moving target”
- Only plan changes when necessary – or optionally adding value
- Plan to give states advance notice to the extent possible
- Work with states to minimize effects

## User Implementation

- Allowed users to start slow
- Specify format preference by message key for each ORI
- XML can be sent via multiple protocols; not limited to web services
- Provide support to users
- Stylesheets, transformations were a necessity because users are and will continue to be at different stages
- As new services are offered, they may only be available via XML and web services

## Typos, other minor mistakes
Do you fix and risk disrupting users or leave incorrect?
Namespaces
Has become considerably more complicated than with initial specifications
New namespaces (example AA)
Multiple GJXDM versions/namespaces
Inheritance – differing requirements for same element
Support of multiple transports
Difficult to utilize SOAP header, WS-Attachments, etc
Hindsight...
Multiple ways to do everything – each with own pros, cons

- Do you fix and risk disrupting users or leave incorrect?
- Namespaces
- Has become considerably more complicated than with initial specifications
- New namespaces (example AA)
- Multiple GJXDM versions/namespaces
- Inheritance – differing requirements for same element
- Support of multiple transports
- Difficult to utilize SOAP header, WS-Attachments, etc
- Hindsight...
- Multiple ways to do everything – each with own pros, cons

**Problems**

## Benefits

- Allows NLETS to share more information with users
- Allows users to customize data presentation via stylesheets
- XML is non-proprietary
- Presents enormous cost savings to states

## Benefits

- Implementing new services quicker and easier
- If a data provider already has info in GJXDM XML, we can quickly position ourselves to receive it and provide to users
- Sharing data within a state is simpler – eliminates the need for XML-to-XML transformations (example: Amber Alert)

## Lessons Learned

- Be sensitive to early adopters
- Recognize that you will need careful change management
- 	(“ripple effect”)
- Consider large issues carefully before beginning
- message structure,
-  	   namespaces, schemas
- Remember that the GJXDM will not provide everything

## Future Plans

- Expand membership and services offered
- Web Services opens new doors to new providers
- Standardize all NLETS transactions in GJXDM-compliant XML

## Questions?

## Thank you

- Kate Silhol
*NLETS Senior Software Engineer*
- ksilhol@nlets.org
- (602) 224-0744
- www.nlets.org