# Presentation: 341660

## Implementor’s Panel:BL’s eJournal Archiving solution using METS, MODS and PREMIS 

- Markus Enders, British Library

**DC2008, Berlin**

## Using METS, PREMIS and MODS for Archiving EJournals

- Digital Library System Program
- Development of a system for ingest, storage and preservation of digital content
- eJournals are the first content stream
- Developing a common format for the eJournal AIP
- Metadata needs:
- Need to understand business processes and data structures
- Structurally complex
- (issues relased in intervals, contain varying number of articles / other publishing matter, submitted in various formats – might vary from article to article within the same issue)
- Production of eJournals is out of control of the digital repository
- No standards for structure of submission packages, file formats, metadata formats, vocabulary

## Using METS, PREMIS and MODS for Archiving EJournals

- Ingest workflow
- SIP (usually packed as zip or tar)
- Contain content files, descriptive metadata files, manifest listings, hashing information for files
- May contain one or several issues; articles for one or several journals
- Structure is different than AIP structure
- File naming conventions representing structure and relationships

## Using METS, PREMIS and MODS for Archiving EJournals

- Ingest workflow: main steps
- Unpack
- Unzip / untar the submitted archive
- Virus check
- Virus check all files
- Normalize
- Normalize content files: NLM.DTD
- Metadata extraction
- create AIP description: descriptive, technical and preservation metadata
- Validation

**Notes:**
Metadata extraction:
Extract information from metadata files provided by the publisher
Extract information from content files (), extract format information (PRONOM database), creating application (if provided)

## Using METS, PREMIS and MODS for Archiving EJournals

- Standardized AIP structure
- Structural relationships, metadata & content is standardized
- Structure depends on technical infrastructure of preservation system
- Metadata Management Component: contains operational metadata
- Archival Store: Write once – supports archival authenticity and track the objects’ provenance
- AIP is stored in the Archival Store

**Notes:**
Structural relationship: title – Issue - article
Uniform search across all digital objects
Guarantee the sustainability of formats, data and structural relationships

## Using METS, PREMIS and MODS for Archiving EJournals

- Granularity of AIP
- Update of AIP: add new package; generations of AIPs need to be managed
- Reasons for updates:
- Migration of content files
- Updates to descriptive metadata
- Updates of other information systems might affect information stored in AIP
- Correction of corrupt content files

**Notes:**
Generation: update to an object
Try to avoid version or edition: overloaded in the library community

Descriptive metadata corrections: small scale (typo), large scale

## Using METS, PREMIS and MODS for Archiving EJournals

- Split logical separated metadata subsets
- Journal, issue, article: one AIP for each
- Can be updated independently
- Structural information is separated from files
- Files are stored in a manifestations (normalized files)
- Five different metadata AIPs representing different kinds of objects
- Each AIP is a separate METS file

**Notes:**
Journal, issue and article AIP only contain metadata: descriptive and administrative metadata
Links between those files

Submission is kept and stored

5 kinds of objects: journals, issues, articles, manifestations, submissions

## Using METS, PREMIS and MODS for Archiving EJournals

- Identifiers
- MMC-ID
- 	Identifier of metadata management component
- 	identifies the intellectual entity
- 	exposed to the outside / external systems
- 	Stored in MODS record
- MMC-ID+
- 	generation dependent MMC-ID,
- 	needed to store relationships between specific generations in a PREMIS record
- DOMID
- 	Identifies a file in the Archival Storage
- 	Identifer stored in Premis record

**Notes:**
MMC-ID: 
Used for linking with ILS

MMC-ID+:
Used in PREMIS records to link to versions

## Using METS, PREMIS and MODS for Archiving EJournals

- Submission
- Describes one submission event
- Records all activities performed during ingest
- Original data as it was provided by the publisher
- Manifestation
- All files necessary for one rendition of an article
- Relationships between those METS files are stored in METS files themselves as well as in Metadata Management Component

**Notes:**
Submission: one submissione vent
All tarred and zipped files

Activities during ingest: submission (transfer), virus check, unpacking

Manifestation: not to be mistaken for FRBR’s definition of a manifestation

## Using METS, PREMIS and MODS for Archiving EJournals

**Notes:**
Journal – issue – Article relationships:
MODS metadata records: <mods:relatedItem type=“host”>
Expressing hierarchical relationships

Journal -> Journal relationships:
<mods:relatedItem type=“preceding”>
Link to predecessor

Issue -> Issue / Article -> Article: relationships:
<mods:relatedItem type=“series”>
Article is part of a series; other parts can be published in other issues (even in other journals)

## Using METS, PREMIS and MODS for Archiving EJournals

**Notes:**
Journal – issue – Article relationships:
MODS metadata records: <mods:relatedItem type=“host”>
Expressing hiearchical relationships

MMCID: identifier of the metadata management component; unique; regarded as persistent

Journal -> Journal relationships:
<mods:relatedItem type=“preceding”>
Link to predecessor

Issue -> Issue / Article -> Article: relationships:
<mods:relatedItem type=“series”>
Article is part of a series; other parts can be published in other issues (even in other journals)

## Using METS, PREMIS and MODS for Archiving EJournals

**Notes:**
Journal – issue – Article relationships:
MODS metadata records: <mods:relatedItem type=“host”>
Expressing hiearchical relationships

Journal -> Journal relationships:
<mods:relatedItem type=“preceding”>
Link to predecessor

Issue -> Issue / Article -> Article: relationships:
<mods:relatedItem type=“series”>
Article is part of a series; other parts can be published in other issues (even in other journals)

## Using METS, PREMIS and MODS for Archiving EJournals

**Notes:**
Generations:
Premis records: <premis:relationshipSubType> generation
Premis event attached to it, event records the reason for a new generation

Manifestation:
Premis records in Manifestation METS files: <premis:relationshipSubType> manifestationOf
But no event attached

## Using METS, PREMIS and MODS for Archiving EJournals

**Notes:**
Manifestation:
Premis records in Manifestation METS files: <premis:relationshipSubType> manifestationOf
But no event attached

Relationship to submission:

ContentFile:
<mets:fptr> file pointer used

Premis events for each file:
integrityCheck, formatIdentification, validation, propertyExtraction
(change: wellformedness as a separate event)

Uncompress event:
Relationship to the archive file (zip or tar)
Archive file must be in the same submission as linked from Manifestation
(redundant data -> refinement of “containedInSubmission” link

Migration Event:
Relationship to source file; file the new was derived from

## Using METS, PREMIS and MODS for Archiving EJournals

**Notes:**
Manifestation:
Premis records in Manifestation METS files: <premis:relationshipSubType> manifestationOf
But no event attached

Relationship to submission:

ContentFile:
<mets:fptr> file pointer used

Premis events for each file:
integrityCheck, formatIdentification, validation, propertyExtraction
(change: wellformedness as a separate event)

Uncompress event:
Relationship to the archive file (zip or tar)
Archive file must be in the same submission as linked from Manifestation
(redundant data -> refinement of “containedInSubmission” link

Migration Event:
Relationship to source file; file the new was derived from

## Using METS, PREMIS and MODS for Archiving EJournals

- PREMIS and MODS metadata are embedded into METS
- Extension schemas
- Premis: <amdSec>
- MODS: <dmdSec>
- Attached to <mets:div>
- Journal, issue, article, manifestation, submission
- PREMIS: representation - object
- PREMIS data in <mets:digiprovMD>
- Attached to <mets:file>
- File only
- PREMIS: file – object
- PREMIS data in <mets:digiprovMD> AND <mets:techMD>

**Notes:**
PREMIS and MODS embedded into METS:
Extension schemas
In different sections – section either attached to a <div> element (title

More provenance metadata besides relationships:
Format information, 
Fixitiy information
Identification

NOT used as publisher’s don’t provided that information:
Creating application
Hardware and software enviroment

## Using METS, PREMIS and MODS for Archiving EJournals

- METS, PREMIS, MODS
- some metadata can be represented in either or several metadata schemas
- Checksums:
- <mets:file CHECKSUM=..../>
- <premis:objectCharacteristics><premis:fixity>
- File size:
- <mets:file SIZE=.../>
- <premis:objectCharacteristics><premis:size>
- Store this information redundantly as they might be used for different purposes

**Notes:**
Like the identifers: some metadata is available in more than one schema

Decision:
Store information redudantly. Used for different purposes:
METS: presentation
PREMIS: perservation; might store more than one CHECKSUM (using different algorythms)

PREMIS records might be used separately in the future 

## Using METS, PREMIS and MODS for Archiving EJournals

- METS, PREMIS, MODS
- some metadata can be represented in either or several metadata schemas
- Format information:
- <mets:file MIMETYPE=..../>
- For display and delivery e.g. via http
- <premis:format>
- Refines the MIMETYPE
- Links to PRONOM database
- For preservation purposes (preservation planing & preservation actions as e.g. migration)

**Notes:**
METS: presentation
MIMETYPE important for delivering an http-response

PREMIS: perservation; link to format registries, refinement of METS MIMETYPE attribute
PRONOM database

## Using METS, PREMIS and MODS for Archiving EJournals

- METS, PREMIS, MODS
- some metadata can be represented in either or several metadata schemas
- Technical Metadata (file):
- Use PREMIS:
- Fixitiy information
- Format
- PREMIS technical information (for files)
- In mets:techMD
- PREMIS non-technical information (for files)
- In mets:digiprovMD

## Using METS, PREMIS and MODS for Archiving EJournals

- METS, PREMIS, MODS
- some metadata can be represented in either or several metadata schemas
- Technical Metadata (file):
- Use PREMIS:
- Fixitiy information
- Format
- Use additional extension schemas for format specific technical metadata (optional) – e.g. rendering & display
- Directly in mets:techMD
- Don’t use MODS <mods:physicalDescription>

## Using METS, PREMIS and MODS for Archiving EJournals

- METS, PREMIS, MODS
- Rights information
- Not intended to be actionable
- Archival, descriptive nature
- Stored in MODS

**Notes:**
Rights information do not support any repository function

According the METS/PREMIS implementation guidelines

## Using METS, PREMIS and MODS for Archiving EJournals

- METS, PREMIS, MODS
- PREMIS events:
- If more than one object (representation or file) is affected, the event is stored in each PREMIS section
- Any attached agent to this event is stored in each  PREMIS section as well
**What kind of events:**
- On file level :
- submission, unCompress, virusCheck, validation, ingest, (wellformness)
- On file level:
- Migration (not yet implemented in software)
- On representation:
- metadataUpdate, (metadataCorrection)

**Notes:**
Rights information do not support any repository function

According the METS/PREMIS implementation guidelines

## Using METS, PREMIS and MODS for Archiving EJournals

- PREMIS 2.0
- Still using premis 1.1; No fundamental changes to data model -> migration is not too difficult, although xml schema it is not backwards compatible
- Extensions to extend PREMIS
- Embed metadata from other schemas into a PREMIS record
- Event outcome, creating application, object characteristics, significant properties: usage needs to be discussed
- objectCharacteristicsExtension: might be useful to store format specific metadata which are only regarded as relevant for preservation purposes

**Notes:**
objectCharacteristicsExtension:
Currently <mets:techMD> is used to store JHOVE output
- Two much output from JHOVE, might only want to store relevant format specific information

## Using METS, PREMIS and MODS for Archiving EJournals

**Conclusion:**
- No single existing metadata schema accommodates the representation of descriptive, preservation and structural metadata.
- Using a combination of of METS, PREMIS and MODS allows us represent eJournal Archival Information Packages in a write-once archival system