# Presentation: 656484

## NARA’s Electronic Records ArchivesProgram

- Ken Thibodeau
- Director
- ERA Program Management Office

*Briefing for Joint Staff and Combatant Commands*
*November 19, 2003*

## NARA’s Electronic Records Challenge

**Take any type of record,**
**Created using any type of application,**
**On any computing platform,**
**From any entity in the Federal Government and any donor.**
**Provide discovery and delivery to anyone with an interest and legal right of access,**
**Now and for the life of the republic**
***Guide and assist other agencies in managing their records***
***For successful transition to e-government***

## Exploiting the Enterprise

**Exploiting the Enterprise**

***Global Information Grid***

***“******Factory to Foxhole”***

***Joint Data Net ***

**WIN-Tactical Network**
**w/ JTRS **

***Comanche***

***Bandwidth ***
***Expansion***

***Predator***

***Global Hawk***

**Unit of Employment**
**(Strategic Level) **

***En-Route Mission***
***Planning***

**UE (XX)**

**UA (X)**

**UE (XXX/XXXX)**

**UA (X)**

***Joint,***
*** ******Interagency, ***
***&***
***Multi-National***
***Interoperability***

**Unit of Employment**
**(Operational Level) **

**Unit of Employment**
**(Strategic Level) **

**Unit(s) of Action**
**(Tactical Level) **

**Unit of Employment**
**(Tactical Level) **

**Future Combat System Network**
**w/ JTRS **

***“******Space to Mud”***

***Home Station***
***Operations Center &***
***Industrial Base***

**Notes:**
Topic 1: Electronic Records.  A. Government Records. (2) Government reliance on electronic records – 1 Military 3
The warfighter is generating “actionable information” – “knowledge” that can be immediately used to the warfighter’s advantage.
Needed knowledge is processed on the Global Information Grid to provide a fused and integrated view that includes collaboration, dynamic ISR tasking, virtual teaming, reach back, and shared awareness.
The warfighter tasks the Global Information Grid to gain better Battlespace awareness using his/her security dynamic profile. The dynamic profile provides information content and services tailored to the warfighters’ basis of knowledge about their preferences and behavior.  For example, the Sensor Fusion Center (an operational node on the Grid that controls ISR assets in the Area of Operations, dynamically tasks additional ISR platforms (through polling available ISR capabilities in the area of operations). 
Shared understanding is the cognitive portion of the Global Information Grid. Shared understanding is at multiple levels:
- Service components at the tactical level are focused on today, while JTF commander is planning to tomorrow.
- Shared understanding can be displayed by an integrated view of the Battlespace (e.g., the CROP), or consist of text and numbers (e.g., success of Effects Based Operations to date, ammunition supply rates, number of casualties).
Information users publish and subscribe information from the Global Information Grid anywhere, anytime using a “Smart Push & Pull.”
Joint military operations must coevolve their business practices, organizations, doctrine, and processes to leverage Net-Centric technology. 

##  A (Federal) Record?

- Information made or received in the course of business
- Kept, or should be kept
- As evidence of functions, organization, policies, decisions, actions, etc.
- Because it contains valuable information
- Regardless of form or characteristics

- An electronic record is a record in a form suitable for processing by a computer

**Notes:**
If we were to imagine ourselves in the year 2015, looking back to today to see what it will take to get us there, we would see future combat systems, mounted formations, seamlessly integrated into the joint force. It will be a decisive Army, trained and equipped for ground dominance and cyber-warfare and exploiting the “high ground” of space – yes, there is a role for the Army in space; we need to participate in that discussion. It will be an army designed from the bottom up around a single, integrated C4ISR architecture – it's time. That architecture will link us to joint, interagency, multi-national, and even back to the legacy and Stryker forces. It'll be a rapidly deployable, mounted formation that could drive into the teeth of an anti-armor fight and prevail by taking down its adversaries. It will be an army with consolidated, streamlined branches and moss – I can hear the component chiefs beginning to listen – streamlined branches and moss composed of warfighters poised to rapidly transition from disaster relief to warfighting – the full spectrum of military operations. The Objective Force and its Future Combat System of systems will be leveraging and delivering with precision the combat power of joint and strategic national assets 
Gen. Eric Shinseki, CSA, Oct 2002, Eisenhower Luncheon

## Does an Electronic Record Have to be Kept in Electronic Form?

- If there is no other way to keep it
- If there is a business need to do so
- E.g., in order to continue to use it in the system in which it was created
- If the evidential or
- informational value
- would be diminished
- by converting it to
- another form.

## Keep a Record In Electronic Form if it has essential properties which cannot exist in any other form.

| Properties | All Records | Electronic Records |
| --- | --- | --- |
| Context |  |  |
| Structure |  |  |
| Content |  |  |
|    Behavior |  |  |

## Slide 7

| Properties | All Records | Electronic Records |
| --- | --- | --- |
| Context | Provenance
Activity | = |
| Structure | Filing system
Document structure | Conceptual & logical data models
Hyperlinks |
| Content |  | Process records
Virtual Reality |
| Behavior | Static, physical inscription
Physical position | Active reproduction
Interaction
Selective, variable presentation |

## A Record in the Form of a Geographic Information System (GIS) Application

- _Data Layers_
- Roads
- Land Use
- Population Density
- Administrative Boundaries
- Evacuation Routes
- Remote Sensing Imagery
- Rivers
- Geography, Geology et al.

- _Behaviors_
- Cartographic Display
- Overlaying of Data Layers
- Feature Selection

## Enduring Values

- Evidence
- Emphasis on Authenticity
- Enhanced by proximity to event, ‘originality,’ quality
- Information
- Emphasis on richness, completeness, etc.
- Enriched by linkage with other information
- Enhanced by authenticity

## Preserving an Electronic Record

- Preservation of electronic records is the end-to-end process which enables re-production of an authentic copy of the record

## Electronic Record Behaviors

## E-transactions

**XML**

## E-transactions: Basic Assumptions

*Different* systems interact in e-transactions.
- The only thing one system needs to know about another is that it can use the same intermediary.
- Anything which must be true about an
- information asset in one system
- must be true about that asset
- in another system
- involved in the e-transaction.

## Electronic Records Archives: Basic Assumptions

- The ERA system must be able to interact with  ***different*** systems.
- At any time, the only thing the ERA system needs to know about another is that it can use the same intermediary.
- Over time, the ERA system cannot know what mediators other systems will use.
- Any record in the ERA system must
- be an authentic copy of that
- record.
- Any record delivered from
- ERA to another system must be
- an authentic copy.

## NARA’s Strategy

- Attack the critical preservation problem
- Make information assets independent of information technology infrastructure
- Define the requirements in terms of the lifecycle management of records
- Find solutions in mainstream technologies being developed to support e-commerce and              e-government
- Align with overall direction of IT in the U.S. Government

## NARA’s Response:The ERA Program

- System acquisition
- The ERA system will
- Support NARA’s end-to-end lifecycle management of all records
- Preserve and provide access to electronic records
- Organizational change management
- Ensure NARA can successfully implement the system
- Research and Exploratory Development
- Address new challenges posed by continuing change in IT and its use in government
- Capitalize on opportunities offered by technological progress

## Partnerships

- NIST

**Notes:**

After we’ve identified the challenge, articulated the vision, and visualized the solution, we needed to move forward toward accomplishing the goal.

Our 1st step was partnering  with industry expert both in the federal and private sector.  NARA recognized that since we had never built a large scale information system and the agency was made up of mostly Archivist, Records Managers, Librarians, and Historians, we needed to learn from those who had much more experience than we did.

- “Strategies to assure long-term preservation of digital records constitute another particularly pressing issue for research....”

## Knowledge-Based Persistent Archives

- Attributes
- Semantics

**Knowledge**

**Information**

**Data**

**Ingest **
**Services**

**Maintenance**

**Access**
**Services**

**Domain to Information Mapping**

**Data Grid**

**Encoding**

**Access Methods**

- Information
- Repository

- Attribute- based
- Query

- Feature-based
- Query

- Knowledge or
- Topic-Based
- Query / Browse

- Knowledge
- Repository for
- Rules

- Relationships
- Between
- Concepts

- Fields
- Containers
- Folders

- Storage
- (Replicas,
- Persistent IDs)

**Notes:**
Encoding
	Data level: TAR, DOS FAT
	Info level: record layout, data element definition
	Knowledge level: Concept maps, models.

Access Mechanisms
	Data level: ftp, “full text search”
	Info level: Z39.5- query for bibliographic information
	Knowledge level: Web portal.

## ERA Virtual Workspace Model

**Notes:**
   At NARA, we are using both the OAIS and InterPARES models to develop the “Electronic Records Archives.” In this effort, we are leveraging major computer and communications research and development activities of many other organizations, including the U.S. National Science Foundation, Defense Advanced Research Projects Agency, Army Research Laboratory, and the National Partnership for Advanced Computational Research (NPACI), including NPACI’s International Affiliate in archival research headquartered at the University of Urbino.
The Electronic Records Archives (ERA) is NARA’s strategic initiative to develop a comprehensive means for preserving electronic records of the U.S. government. A critical feature that sets ERA apart is that it explicitly addresses the archival desiderata listed previously, notably incorporating the expectation of change in the architecture of the system.
This conceptual model shows that ERA will comprise 3 virtual ‘work spaces’ corresponding to the basic OAIS functions. There are 3 important ideas behind the virtual work space concept: 1. Each will have built-in functionality to support core processes. 2.  Each work space will interact with the others, and with other systems, through software mediators. 3. Each work space will enable use of specialized tools when needed.  The last two features are designed to enable harware and software components to be replaced with minimal impact on the system as a whole or on the records being preserved.

## Release RFP: Current Quarter
Contract for Competing Designs:
Q 3 FY 2004
Select single developer 
Q3 FY 2005
Initial Operating Capability: 
FY 2007
Full Operating Capability 
FY 2011

- Contract for Competing Designs:
- Q 3 FY 2004
- Select single developer
- Q3 FY 2005
- Initial Operating Capability:
- FY 2007
- Full Operating Capability
- FY 2011

- Acquisition of the ERA System

**Notes:**
Paths: Information Assets in E-government
The challenge of preserving digital assets
NARA’s response to the challenge

## To Manage Electronic Records Efficiently and Effectively 

- Automate processing
- Incorporate laws, regulations, policies, and sound archival and records management principles in the system.
- Make electronic records *self-describing* and *self-validating* to facilitate processing and management

## Self-presenting documents 

## Plain text view of self presenting Document

## Self-describing Electronic Records 

## Sample Self-Describing Document

**<Document type:** transmittal memorandum**>**
**<Document Date:> **February 10, 1998**</>  <Directive Transmitted:> **Circular A-119**</> <Addressees:>**Heads Of Executive Departments And Agencies**</> <Subject:>**Federal Participation in the Development and Use of Voluntary Consensus Standards and Conformity Activities **</> < Message Body:>**Revised OMB Circular A-119 establishes policies on federal use

## Automated Controls: “Template”

- An abstraction or articulation of the properties of a record, set of records, or a transaction involving records, which must be controlled.
- The template for all records requires that the provenance, date, and archival bond be specified.
- The template for State Dept. diplomatic messages requires that all content be plain text.
- The template for all transfers of records to the National Archives is the S.F. 258

## Self-validating Electronic Records 

- A **self-describing document** has meaningful indicators of what the document is and/or contains
- Ideally, the indicators (“tags”) are simple and clear enough to be correctly interpreted by any computer or by a person
- When the tags can be used to compare a self-describing document to a model (“template”) of what it should be, it can be a **self-validating document**

## Template Example

- A **“**Transmittal Memorandum” must contain
- Reference to the directive which it transmits
- A subject header
- Indication of addressees
- Identity of the transmitting official
- Date transmitted by the official
- A message body
- The message body may identify the legal basis and one or more action officers

**Message Body**

**Document Date**

**Document Type: **Transmittal Memorandum

**Addressees**

**Subject**

**Author**

**Status**

**Directive**

- If the template is in executable form, the computer can determine automatically if a record is what it should be.

## Geography Markup Language

## Hierarchy of Controls

**Tier I: ****NARA STANDARDS**
- Apply to Abstract Classes of Records and Sets of Records and to Lifecycle Transactions
- Create:    Preserver
- Register:  Preserver

**Tier II: Top Level Producer Templates**
- Apply to either abstract or real Classes &
- Sets of Records
- Create:  Record Producer
- Register:  Appraiser

- Conform to NARA standards

- Conform to Tier II templates

**Tier III: Subsidiary Producer Templates**
- Apply to real Classes of Records and Sets of Records
- Create:    Record Producer
- Register:  Record Producer

## Records Template Hierarchy

## Files Template Hierarchy

**NARA STANDARDS**

**Files Template Hierarchy**

***DoD***
***OMPF ***

***NIH ***
***Grant Files***

**Tier III**

***Army***
***OMPFs***

**Tier I**

***Project ***
***Grant ***

***Center***
***Grant***

***Navy***
***OMPFs***

***File***

***Case File***

***Subject File***

***NIH ***
***Medical Records***

***Chron File***

***Air Force***
***OMPFs***

***Masrines***
***OMPFs***

## Case File Standard

- Required Elements
- Case Identifier
- Subseries of
- Nature of case
- Case Opening Record
- Case Closing Record
- Case Start Date
- Case End Date
- Disposition Authority
- ....

- Optional Elements
- Producer Required Records
- Producer Optional Records
- Producer Prohibited Records
- Cross Reference
- Unspecified Records
- ....

***File***

***Case File***

- File
- Identifier
- Location
- Coverage
- Required Records
- Dates
- Disposition Authority

## Case File: d-OMPF Example

**Header:** SSN=xxx-xx-xxxx
- Doc Class=DD 214
- Record Date=*dd/mm/yyyy*
*...**.*

**Record**

- File

## Virtual DD 214 Database Logical Model

- Person Document

- Person Document Role

- Person

- Record Representation

- Document Representation

- Certification

- Representation Status

- Person Document Representation

- Person Document Representation Reason

- Pagination

- Organization Document Representation

- Organization

- Organization Document Representation Reason

- Document Identification

- Document Segment

- Document Segment Association

- Organization Document

- Organization Document Role

- Document

- Document Type

- Document Status

- Position Document

- Position

## OMPF Template Hierarchy

- DD 214

|  |  |  |  |
| --- | --- | --- | --- |
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |