# Presentation: 290347

## An Introduction to Preservation Metadata and the PREMIS Data Dictionary

- Rebecca Guenther, Library of Congress
**ALA Midwinter 2009**
**Intellectual Access to Preservation Metadata Interest Group**
**Sat. January 24, 2009**

## Overview

- What is preservation metadata?
- Development of PREMIS
- PREMIS data dictionary
- Features of the data dictionary
- Implementing PREMIS
- PREMIS Maintenance Agency
- Future directions

## Digital preservation: advances & remaining challenges

- Groups around the world and conferences continue to make significant progress in raising awareness about digital preservation imperative
- Gradual shift in focus from **articulating*** *problem to **solving*** *it* *...
- Not so much *“Why is digital preservation important” *anymore; rather, *“What must be done to achieve preservation objectives?”*
- Many practical challenges in **implementing** reliable, sustainable digital preservation programs
- One key **implementation** challenge: preservation metadata

**Notes:**
A critical need ... preservation metadata is key component of digital preservation infrastructure
As we know, metadata is “Structured information that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource”
Preservation metadata is an important subset of this which “supports and documents the digital
preservation process”

Reinforced by earlier presenters:
Manjula Patel: DCC believes “Representation information is the key to long-term access”
John Kunze spoke of technical information being extracted before dessicating versions because it does provide clues.
Stephen Abrams: “Effective digital preservation requires the proper characterization of the objects [under managed care]  Fundamental characterization property is format.”

## Preservation metadata includes:

**Preservation metadata includes:**

- Provenance:
*Who has had custody/ownership of the **digital object?*
- Authenticity:
*Is the digital object what it purports to be?*
- Preservation Activity:
*What has been done to preserve it?*
- Technical Environment:
*What is needed to render and use it?*
- Rights Management:
*What IPR must be observed?*
**Makes digital objects self-documenting across time**

**Content**

**Preservation**
**Metadata**

**10 years on**

**50 years on**

**Forever!**

**Notes:**
There are a number of questions that need to be answered to ensure preservation of digital objects over time. We all know that files can get corrupted and unreadable and it is important to be able to track who has had custody of an object and all actions performed on an object. What preservation strategies have been applied needs to be recorded as well as what software or hardware is needed to use it and whether there are any restrictions on preservation activities (in addition to access). If metadata is associated with an object it can become self-documenting over time.

## What does PREMIS cover?

- Administrative metadata that supports the digital preservation process
- Provides information to help manage a resource for preservation purposes
- Technical characteristics
- Information about actions on an object
- Relationships (structural and derivative)
- Structural: indicates how compound objects are put together
- Derivative: results of common preservation actions
- Rights metadata associated with preservation

**Notes:**
PREMIS stands for PREservation Metadata: Implementation Strategies Working Group, which developed a data dictionary of elements needed to support the preservation of digital objects. Preservation metadata falls into various categories, each of which also support other uses (such as access). 

## Some background ...

**Pre-2002:** various preservation metadata element sets released
- Different scopes, purposes, underlying models/assumptions
- No international standard; little consolidation of expertise/best practice
**June 2002:** Preservation Metadata Framework
- International working group (jointly sponsored by OCLC, RLG)
- Comprehensive, high-level description of types of information constituting preservation metadata
- Used OAIS reference model as starting point
- Set of “prototype” preservation metadata elements
- Consensus-based foundation for developing formal preservation metadata specifications ... but not an “off-the-shelf, ready to implement” solution
**Post-2002: **Need implementable preservation metadata, with guidelines for application and use, relevant to a wide range of digital preservation systems and contexts
- Motivated formation of **PREMIS Working Group**

## PREMIS Working Group

- June 2003: OCLC, RLG sponsored new international working group:
***PREMIS: Pre****servation ****M****etadata: ****I****mplementation ****S****trategies*
- Membership:
**> 30 experts from 5 countries, representing libraries, museums, archives, government agencies, and the private sector**
- Co-Chairs: Priscilla Caplan (FCLA), Rebecca Guenther (LC)
- Objective 1: Identify and evaluate alternative strategies for encoding, storing, managing, and exchanging preservation metadata
- PREMIS Survey Report (September 2004)
- Snapshot of current practices/emerging trends related to managing and using preservation metadata in digital archiving systems
- http://www.oclc.org/research/projects/pmwg/surveyreport.pdf
- Objective 2: Define implementable, core preservation metadata, with guidelines/recommendations for management and use

## PREMIS Data Dictionary

**May 2005: ***Data Dictionary for Preservation*
*	**Metadata: Final Report of the PREMIS Working Group*
**March 2008: ***PREMIS Data Dictionary for Preservation *
*	**Metadata, version 2.0*
- Includes PREMIS Data Dictionary, context/assumptions, data model, usage examples
- XML schema to support implementation
**Data Dictionary: **
- Comprehensive view of information needed to support digital preservation
- Guidelines/recommendations to support creation, use, management
- Based on deep pool of institutional experiences in setting up and managing operational capacity for digital preservation

**http://www.loc.gov/standards/premis/v2/premis-2-0.pdf**

**Notes:**
Current version is now 2.0. 

## 2005 British Conservation Awards: Digital  Preservation Award

**2005 British Conservation Awards: Digital  Preservation Award**

**2006 Society of American Archivists Preservation Publication Award**

**Notes:**
The PREMIS Working Group, which developed the first version of the PREMIS Data Dictionary, has won 2 major preservation awards.

## Some guiding principles ...

- “Implementable, core, preservation metadata”:
- “Preservation metadata”: maintain viability, renderability, understandability, authenticity, identity in a preservation context
- “Core”: What most preservation repositories need to know to preserve digital materials over the long-term
- “Implementable”: rigorously defined; supported by usage  guidelines/recommendations; emphasis on automated workflows
- “Technical neutrality”:
- Digital archiving system: no assumptions about specific archiving technology, system/DB architectures, preservation strategy
- Metadata management: no assumptions about whether metadata is stored locally or in external registry; recorded explicitly or known implicitly; instantiated in one metadata element or  multiple elements
- Promotes flexibility, applicability in wide range of contexts

## Scope of data dictionary

- Implementation independent
- Descriptive metadata out of scope
- Technical metadata applying to all or most format types
- Media or hardware details are limited
- Business rules are essential for working repositories, but not covered
- Rights information for preservation actions, not access

**Notes:**
The PREMIS Data Dictionary assumes that implementation decisions need to be made and does not dictate a syntax, although an XML schema is provided.
Descriptive metadata only includes an identifier, but otherwise is out of scope, although its importance is recognized.
Technical metadata that is included is only that which applies to all format types, not specific file format types (e.g. image, video, etc.)
Business rules are policy decisions that a repository makes about preservation strategies that may or may not be relevant to another institution. Some examples are retention periods or risk assessment.

## What PREMIS is and is not

- What PREMIS is:
- Common data model for organizing/thinking about preservation metadata
- A checklist for core metadata in a repository
- Guidance for local implementations
- Standard for exchanging information packages between repositories
- What PREMIS is not:
- Out-of-the-box solution: need to instantiate as metadata elements in repository system
- All needed metadata: excludes business rules, format-specific technical metadata, descriptive metadata for access, non-core preservation metadata
- Lifecycle management of objects outside repository
- Rights management: limited to permissions regarding actions taken within repository

**Notes:**
PREMIS may be used in various ways. There is an emphasis on its usefulness as an exchange format between repositories, where a standard is clearly needed. Internally, it may serve as a checklist for core metadata included in the repository.
It needs to be used in conjunction with other metadata and is only one part of the infrastructure needed for digital preservation.

## PREMIS Data Model

**Intellectual**
**Entities**

**Objects**

**Rights****Statements**

**Agents**

**Events**

**Notes:**
These are the relationships in the PREMIS data model. Although descriptive metadata is out of scope, objects can point to descriptions of intellectual entities or the entity itself, in either direction. From the data model you see that agents (people, organizations or software) can only act on objects through events or rights, not directly. Objects can have relationships to other entities in the data model or to each other. 

## Intellectual Entities

- Examples:
- Rabbit Run by John Updike  (a book)
- “Maggie at the beach”
- 	(a photograph)
- The Library of Congress Website (a website)
- The Library of Congress: American Memory Home page (a web page)

- Set of content that is considered a single intellectual unit for purposes of management and description (e.g., a book, a photograph, a map, a database)
- May include other Intellectual Entities (e.g. a website that  includes a web page)
- **Has one or more digital representations**
- Not fully described in PREMIS DD, but can be linked to in metadata describing digital representation

## Objects

- Examples:
- chapter1.pdf (a file)
- chapter1.pdf + chapter2.pdf + chapter3.pdf (representation of a book w/3  chapters)
- TIFF file containing header and 2 images (2 bitstreams (images), each with own set of properties (semantic units): e.g., identifiers, technical metadata, inhibitors, ... )

- Discrete unit of information in digital form
- **Objects are what repository actually preserves**
- Three types of Object:
**FILE:** named and ordered sequence of bytes that is known by an operating system
**REPRESENTATION:** set of files, including structural metadata, that, taken together, constitute a complete rendering of an Intellectual Entity
**BITSTREAM:** data within a file with properties relevant for preservation purposes (but needs additional structure or reformatting to be stand-alone file)

## Object Example 1: photo in two formats

## Object Example 2: book in two versions

## An important aside about Objects	

- Repository does NOT have to control Objects at all levels
- E.g., repository may only manage files, not representations or bit streams.
- The PREMIS DD tells you:
**IF** you control at the representation level, these are the semantic units (properties) that pertain to representations;
**IF** you control at the file level, these are the semantic units (properties) that pertain to files;
**IF** you control at the bitstream level, these are the semantic units (properties) that pertain to bit streams;
**AND IF** you control at multiple levels, you need to record relationships between them

## Events

- Examples:
- Validation Event: use JHOVE tool to verify that chapter1.pdf is a valid PDF file
- Ingest Event: transform an OAIS SIP into an AIP
- Migration Event: create a new version of an Object in an up-to-date format

- An action that involves or impacts at least one Object or Agent associated with or known by the preservation repository
- Helps document digital provenance.  Can track history of Object through the chain of Events that occur during the Objects lifecycle
- Determining which Events are in scope is up to the repository (e.g., Events which occur before ingest, or after de-accession)
- Determining which Events should be recorded, and at what level of granularity is up to the repository

## Agents

- Examples:
- Priscilla Caplan (a person)
- Florida Center for Library Automation (an organization)
- Dark Archive in the Sunshine State implementation (a system)
- JHOVE version 1.0 (a software program)

- Person, organization, or software program/system associated with an Event or a Right (permission statement)
- Agents are associated only indirectly to Objects through Events or Rights
- Not defined in detail in PREMIS DD; not considered core preservation metadata beyond identification

## Rights Statements

- Example:
- Priscilla Caplan grants FCLA digital repository permission to make three copies of  metadata_fundamentals.pdf for preservation purposes.

- An agreement with a rights holder that grants permission for the repository to undertake an action(s) associated with an Object(s) in the repository.
- Not a full rights expression language; focuses exclusively on permissions that take the form:
- Agent X grants Permission Y to the repository in regard to Object Z.

## Semantic units pertaining to objects: technical metadata

- objectIdentifier
- preservationLevel
- significantProperties
- objectCategory
- objectCharacteristics
- creatingApplication
- originalName
- storage
- environment

- signatureInformation
- relationship
- linkingEventIdentifier
- linkingIntellectual Entity Identifier
- linkingPermission StatementIdentifier

## Semantic units pertaining to Events: provenance and preservation activity

- eventIdentifier
- eventType
- eventDateTime
- eventDetail
- eventOutcome
- eventOutcomeDetail
- linkingAgentIdentifier
- linkingObjectIdentifier

## Semantic units pertaining to Rights

- rightsStatement
- rightsStatement Identifier
- rightsBasis
- copyrightInformation
- licenseInformation
- statuteInformation

- rightsGranted
- act
- restriction
- termOfGrant
- rightsGranted
- linkingObjectIdentifier
- linkingAgentIdentifier
- rightsExtension

## Semantic units pertaining to Agents

- agentIdentifier
- agentName
- agentType

## Data dictionary descriptions

| 
Semantic unit | Name that is descriptive and unique. Use externally aids interoperability. Need not be used internally in repository. |
| --- | --- |
| Semantic components | If a container, lists its sub-elements. Each component has own entry.  |
| Definition | Meaning of semantic unit |
| Rationale | Why the unit is needed (if not obvious) |
| Data constraint | How it should be encoded; ”Container”: an umbrella for two or more; no values given”None”: can take any form”Value should be taken from a controlled vocabulary” |
| Object category | Representation                                  File                                      Bit stream |
| Applicability | Whether it applies to the category of object |
| Examples | Illustrative examples of values |
| Repeatability | Whether it can take multiple values |
| Obligation | Whether values must be given.”Mandatory”: something the repository must know independent of how or whether the repository records it. Means mandatory if applicable. If not explicitly recorded, it must be provided in exchange. |
| Creation/maintenance notes | Information about how values may be obtained or updated. |
| Usage notes | Information about intended use.  |

**For each**
**level of**
**Object**

## Sample Data Dictionary entry

## Community interest

- PREMIS Data Dictionary product of **collaboration** and **consensus**
- PREMIS membership reflects variety of institutions, domains, countries
- Multiplicity of perspectives promotes applicability in multiplicity of contexts
- Digital preservation is a shared problem; this invites shared solutions
***Data Dictionary useful to any institution or organization committed to the long-term preservation of digital materials***

## PREMIS Maintenance Activity

- Web site:
- Permanent Web presence, hosted by
- 		Library of Congress
- Central destination for PREMIS-related
- 		info, announcements, resources
- Home of the PREMIS Implementers’ Group (PIG) discussion list
- PREMIS Editorial Committee:
- Set directions/priorities for PREMIS development
- Coordinate future revisions of Data Dictionary and XML schema
- Promote implementation
- Membership: Library of Congress, OCLC, FCLA, British Library, Library and Archives Canada, BStU (Germany), MIT/DSpace

**http://www.loc.gov/standards/premis/**

**Notes:**
Membership has changed; previously we had a representative from Los Alamos National Labs, National Archives in Scotland and National Library of Australia.

## Activities

- First revision of Data Dictionary (PREMIS 2.0)
- Revised by EC and released in late March
- Changes based on extensive discussions with implementers and on Editorial Committee
- Guidelines for using PREMIS with METS (draft available at:)
- _[http://www.loc.gov/premis/guidelines-premismets.html](http://www.loc.gov/premis/guidelines-premismets.html)_
- PREMIS Implementers’ Registry
- _[http://www.loc.gov/premis/premis-registry.html](http://www.loc.gov/standards/premis/premis-registry.html)_
- Developing controlled vocabularies in LC registry
- PREMIS Tutorials:
- Previous: Glasgow, Boston, Stockholm, Albuquerque, Washington, San Diego
- Upcoming: Rome in Feb.
- Current consultancies:
- Understanding PREMIS
- Tool for converting PREMIS to METS/PREMIS and vice versa

**Notes:**
LC has contracted with Florida Center for Library Automation for these 2 products:
An introduction to PREMIS document
 A tool that can convert a PREMIS only instance to a PREMIS in METS instance and a METS with PREMIS to a PREMIS only instance

## Some implementers ...

**DAITTSS (Florida):** a preservation repository for the use of the libraries of the public universities of Florida. Uses a locally-developed software application (DAITSS), which implements most of the PREMIS data elements.
**Ex Libris (DigiTool): **an enterprise solution for the management of digital assets in libraries and academic environments consisting of a number of modules, each designed to address different needs, functions, and workflows pertaining to the life cycle of a digital object
**British Library electronic journal archiving project: **uses METS, MODS, PREMIS for information packages** **
- For more information see:
- http://www.loc.gov/premis/premis-registry.html

## Why is PREMIS important to catalogers?

- As we take responsibility for more digital materials, we need to ensure that they can be used in the future
- Most preservation metadata will be generated from the object, but catalogers may need to verify its accuracy
- Catalogers may need to play a role in assessing and organizing digital materials
- Understanding the structure of complex digital objects
- Determining significant properties that need to be preserved

**Notes:**
Catalogers have always done some assessment of material. Also, there are data elements that serve a function both for the key user tasks of bibliographic (descriptive) records and for preservation purposes. 
Although much of the PREMIS metadata will be generated from the object (e.g. format, size, and other information in file headers that get there at the creation of the file), there may be a need for catalogers to verify its accuracy.  There is much discussion about something called “significant properties” that are special features of the object that need to be preserved over time, and this is where it is important for an assessment to be made, and catalogers may need to play a role. An example might be a PDF file that has an embedded graph where the color of the lines determine the meaning. In this case, it is important that the color be preserved, and someone would need to look at the object to make the determination and state it in the metadata.

## Conclusions

- PREMIS Data Dictionary provides critical piece of reliable digital preservation infrastructure comprised of technology, standards, and best practice
- PREMIS was produced from an international, cross-domain, consensus-building process and is applicable to any preservation effort
- PREMIS Data Dictionary is a building block with which effective, sustainable digital preservation strategies can be implemented
- PREMIS Data Dictionary tightly focused on implementation:
- Practical implementation was guiding principle in all discussions
- Developed tools (XML schema) to support implementation; released with Data Dictionary
- Further work with encouragement for international participation and tools development is ongoing
**Unglamorous but necessary infrastructure!**

**Notes:**
PREMIS Data Dictionary is the first comprehensive technical specification for preservation metadata produced from an international, cross-domain, consensus-building process

Recognized need for evidence of long-term commitment:
Set up PREMIS Maintenance Activity to support Data Dictionary release
Permanent Web presence hosted by US Library of Congress
Central destination for PREMIS information, announcements, resources
Discussion list for PREMIS implementers
Coordinate future revisions of Data Dictionary and XML schema
OCLC, RLG, LC, FCLA currently administering Maintenance Activity; governance structure to be released soon; participation open to all

PREMIS has contributed preservation metadata resources to the digital preservation community that are:
Immediately and freely available
Oriented toward practical implementation
Supported by implementation tools (XML schema) 
Supported by a long-term commitment (PREMIS Maintenance Activity)

## URLs, etc.

**PREMIS Maintenance Activity:**
- 	http://www.loc.gov/standards/premis/
***PREMIS Data Dictionary for Preservation Metadata:***
- 	_[http://www.loc.gov/standards/premis/v2/premis-2-0.pdf](http://www.loc.gov/standards/premis/v2/premis-2-0.pdf)_
**PREMIS Implementation Registry**
- _[http://www.loc.gov/standards/premis/premis-registry.php](http://www.loc.gov/standards/premis/premis-registry.php)_
**PREMIS Implementers Group list**
- _[http://listserv.loc.gov/listarch/pig.html](http://listserv.loc.gov/listarch/pig.html)_
**	**