Overviewing Digital Preservation:
Facets for Long-term Use of Digital
Resources and Archives
Shigeo Sugimoto
Faculty of Library, Information and Media Science
University of Tsukuba
sugimoto@slis.tsukuba.ac.jp

2018/8/23

1

New Title of this talkrevised based on conversation with Profs. Hara and Kameda in a car
heading toward a dinner place yesterday evening

Metadata Issues for Long-term
Use of Databases acrossCommunities
2018/8/23

2

Outline• First half of this talk is intended to share ideas and models forlongevity of digital archives and digital resources, i.e. digitalpreservation, among the participants• Second half is prepared to share basic concepts of metadata designfor better interoperability across communities and over time anddiscussions on metadata design for sharing information about digitalarchives/databases

2018/8/23

3

Part 1 Digital Preservation – an Overview• Digital Entities on Information Environment• Digital Preservation as Maintenance Activity• Bird’s Eye View of Digital Archives – Facets of Digital Preservation

2018/8/23

4

Digital Entities on Information Environment
Digital Instances

User Environments

Concepts, Rights, etc.
Digital Documents and Resourcestiedformat

File
Binary dataUsersPrintDisplay

Access

DisplayFile
2018/8/23

5

Digital Entities on Information Environment
Digital Instances

User Environments

Users

Concepts, Rights, etc.
Digital Documents and Resources

Print
All instances
areexpressedasa
digitalentitytied
AccessFileontheInternet,e.g.,books,files,digitalformatDisplay
Binary data
contents, users, user environments,concepts, etc.Display

File
2018/8/23

6

Digital Entities and Metadata on Web
metadata

Digital Instance

Users

User Environment

metadata
Digital Documents …

tiedformat

Print

File
Binary data

Display

Access

DisplayFile
2018/8/23

7

Digital Entities and Metadata on Web
metadata

Digital Instance

Users

User Environment

metadata
Digital Documents …

tiedformat

Print

File
Binary data

Display

AccessMetadata is a key component to access/use
Display
resources on the Internet
File
2018/8/23

8Digital Entities and Metadata on Web
metadata

Digital Instance

Users

User Environment

metadata
Digital Documents …

tiedformat

Print

File
Binary data

Display

Access

DisplayFile
2018/8/23

9

Digital Entities and Metadata on Web
metadata

Digital Instance

Users

User Environment

metadata
Digital Documents …

Print

No resource
isFilefindable/accessible
tiedformat
Display
If we lose metadata

AccessBinary dataDisplay
File
2018/8/23

10

Digital Entities and Metadata on Web
metadata

Digital Instance

Users

User Environment

metadata
Digital Documents …

tiedformat

Print

File
Binary data

Display

Access

DisplayFile
2018/8/23

11

Metadata Longevity in Digital Preservation
metadata

Digital Instance

Users

User Environment

metadata

Digital Documents …
Print asWe
need to keep metadata alivetied
Filewellformat
as primaryBinary
objects
for longevity
Display ofdata
digital archives and digital resources

Access

DisplayFile
2018/8/23

12

Digital Preservation as Maintenance• Digital Preservation: Keep digital resources usable over time
• In other words, Long-term Use and Longevity of digital resources• Some Questions• What is “(digital) resource” – medium or content?
• What is the difference between “digital resource” and “non-digital (or analog)resource”?
• What does “keep resources usable” mean – do we have to keep a resource usable as
it is or can we modify it?
• “as is”  bit level or functionality level?
• “modifiable”  what features have to be preserved – visual (fonts, layout, etc.), functionality(look-and-feel such as page flipping and input mode), etc.

• Thus, lots of questions  No single answer• Preservation policy and strategy of an institution depend on their needs
2018/8/23

13

Digital Preservation as Maintenance• “How many years do we need to keep digital resources alive?” is afrequently asked question but we don’t need to answer this question
directly, because• all memory institutions would say “we keep the resources forever / aslong as our institution exists”• What we can do is to hand over the resources to our next generation• We need to properly maintain the resources in accordance with ourgiven social environments and technologies, which may change
generation to generation• Digital preservation is to maintain digital resources usable for long-term long-term maintenance of digital resources
2018/8/23

14

Digital Preservation – a Bird’s Eye View
• Overview of digital preservation• Stake holders and functional components• Digital preservation is “management” and need engineering point of view

• Digital Preservation is not just a longevity of digital information media
but has various facets. This section will overview digital preservationfrom a generalized viewpoint• This section will discuss metadata for digital preservation because
metadata is a key component for digital preservation.

2018/8/23

15

Functional Entities and Stake Holders in Digital Preservation
Stake Holders
MemoryInstitutions(GLAMs)

DigitalConservation

Governments,
Companies

Packaged
Resource
Preservation

DigitalArchives

Universities, ResearchInstitutions. etc.

Publishers, Movie
industries, etc.

Long-Life Active
Datasets

Digital Preservation
Planning (Assistance)
Services

Metadata Schema Information Services(for Digital Resource Longevity)
Identifier
Persistency andMaintenanceFile Format
Registry
Metadata Schema Standard Registry

Digital Preservation
Management (Assistance)
ServicesGeneral
PublicRights
ManagementServices

OAIS-based Information
Package Repository

Trusted Bit-Level Preservation Repository

2018/8/23

Functional Entities

16

Digital Preservation – a Bird’s Eye View
• Technological Infrastructure• Trusted Bit-Level Preservation
Repository• File Format Registry• Metadata Schema Standard Registry• Identifier Persistency andMaintenance• Metadata Schema Information
Services (for digital longevity)• Individual Content Preservation
Technologies• Digital Conservation• Packaged Resource Preservation
2018/8/23• Long-term Data Services• Digital Archives• Long-Life Active Datasets• Digital Preservation Services• Digital Preservation Planning(Assistance) Services• Digital Preservation Management(Assistance) Services• OAIS-based Information Package
Repository• Social Infrastructure• Rights Management Services
17Functional Entities and Stake Holders in Digital Preservation
Stake Holders
MemoryInstitutions(GLAMs)DigitalConservation

Governments,
Companies

Packaged
Resource
Preservation

DigitalArchives

Universities, ResearchInstitutions. etc.

Publishers, Movie
industries, etc.

Long-Life Active
Datasets

Digital Preservation
Planning (Assistance)
Services

Metadata Schema Information Services(for Digital Resource Longevity)
Identifier
Persistency andMaintenanceFile Format
Registry
Metadata Schema Standard Registry

Digital Preservation
Management (Assistance)
ServicesGeneral
PublicRights
ManagementServicesOAIS-based Information
Package Repository

Trusted Bit-Level Preservation Repository

2018/8/23

Functional Entities

18Functional Entities and Stake Holders in Digital Preservation
Stake Holders
MemoryInstitutions(GLAMs)

DigitalConservation

Governments,
Companies

Packaged
Resource
Preservation

DigitalArchives

Universities, ResearchInstitutions. etc.

Publishers, Movie
industries, etc.

Long-Life Active
Datasets

Digital Preservation
Planning (Assistance)
Services

Metadata Schema Information Services(for Digital Resource Longevity)
Identifier
Persistency andMaintenanceFile FormatRegistry
Metadata Schema Standard Registry

Digital Preservation
Management (Assistance)
ServicesGeneral
PublicRights
ManagementServicesOAIS-based Information
Package Repository

TechnologicalInfrastructure
Trusted Bit-Level Preservation Repository

2018/8/23

Functional Entities

19Functional Entities and Stake Holders in Digital Preservation
Individual ContentMemory
Preservation
Institutions
Technologies(GLAMs)DigitalConservation

Stake Holders
Governments,
Companies

Packaged
Resource
Preservation

DigitalArchives

Universities, ResearchInstitutions. etc.

Publishers, Movie
industries, etc.

Long-Life Active
Datasets

Digital Preservation
Planning (Assistance)
Services

Metadata Schema Information Services(for Digital Resource Longevity)
Identifier
Persistency andMaintenanceFile FormatRegistry
Metadata Schema Standard Registry

Digital Preservation
Management (Assistance)
Services

GeneralPublicRights
ManagementServices

OAIS-based Information
Package Repository

Trusted Bit-Level Preservation Repository

2018/8/23

Functional Entities

20

Functional Entities and Stake Holders in Digital Preservation
MemoryInstitutions(GLAMs)DigitalConservation

Governments,
Companies

Packaged
Resource
Preservation

Stake Holders
Long-term Data
Services
Universities, ResearchDigital
Archives

Institutions. etc.

Publishers, Movie
industries, etc.

Long-Life Active
Datasets

Digital Preservation
Planning (Assistance)
Services

Metadata Schema Information Services(for Digital Resource Longevity)
Identifier
Persistency andMaintenanceFile FormatRegistry
Metadata Schema Standard RegistryDigital Preservation
Management (Assistance)
Services

GeneralPublicRights
ManagementServices

OAIS-based Information
Package Repository

Trusted Bit-Level Preservation Repository

2018/8/23

Functional Entities

21

Functional Entities and Stake Holders in Digital Preservation
MemoryInstitutions(GLAMs)DigitalConservation

Governments,
Companies

Packaged
Resource
Preservation

Stake Holders
Digital Preservation
Services
Universities, ResearchDigital
Archives

Institutions. etc.

Publishers, Movie
industries, etc.

Long-Life Active
Datasets

Digital Preservation
Planning (Assistance)
Services

Metadata Schema Information Services(for Digital Resource Longevity)
Identifier
Persistency andMaintenanceFile Format
Registry
Metadata Schema Standard Registry

Digital Preservation
Management (Assistance)
Services

GeneralPublicRights
ManagementServices

OAIS-based Information
Package Repository

Trusted Bit-Level Preservation Repository

2018/8/23

Functional Entities

22

Functional Entities and Stake Holders in Digital Preservation
Stake Holders
MemoryInstitutions(GLAMs)DigitalConservation

Universities, ResearchInstitutions. etc.

Governments,
Companies

Packaged
ResourcePreservation

DigitalArchives

Long-Life Active
Datasets

Metadata Schema Information Services(for Digital Resource Longevity)
Identifier
Persistency andMaintenanceFile FormatRegistry
Metadata Schema Standard Registry

GeneralPublishers, Movie Social Infrastructure
Publicindustries, etc.
Digital Preservation
Planning (Assistance)
Services
Digital Preservation
Management (Assistance)
Services

RightsManagementServices

OAIS-based Information
Package Repository

Trusted Bit-Level Preservation Repository

2018/8/23

Functional Entities

23

Part 2 Metadata Issues• Metadata in Digital Archives• Metadata Interoperability

2018/8/23

24

Functional Entities and Stake Holders in Digital Preservation
MemoryInstitutions(GLAMs)

Governments,
Companies

DigitalConservation

Packaged
Resource
Preservation

Stake Holders
Long-term Data
Services
Universities, ResearchDigital
Archives

Institutions. etc.

Publishers, Movie
industries, etc.

Long-Life Active
Datasets

Digital Preservation
Planning (Assistance)
Services

Metadata Schema Information Services(for Digital Resource Longevity)
Identifier
Persistency andMaintenanceFile FormatRegistry

Digital Preservation
Management (Assistance)
ServicesGeneral
PublicRights
ManagementServices

OAIS-based Information
Package Repository

SchemaonStandardRegistry
Rest of this talkMetadata
focuses
metadata
issues for “LongTerm Data Services Issue”
Trusted Bit-Level Preservation Repository

2018/8/23

Functional Entities

25

Digital Space

Digital Space
Curated Digital
Instance

Curated Digital
Instance

Metadata in Digital Archives

member-of/ aggregates

member-of/ aggregates

Collected Set of Digital
Resources

Collected Set of Digital
Resources

Digital
CH objects

DigitalCH objects

DigitalObject

DigitalObjectconverted-to/ converted-fromconverted-to/ converted-fromCollected Set of
Non-Digital Resources

Collected Set of
A GeneralizedofResources
Digital Archives from a paperby Chiranthi
Wijesundara,
aggregated-as/
aggregated-as/ Model
Non-Digital
Non-Digital
related-to
related-to
ObjectWinda Monika,&ShigeoSugimoto
published
atICADL2017
Non-DigitalObjectrepresented-by/ represents

Intangible
performs
Tangible
Clarify entities included in organization
processesofdigitalarchivesoftangible
Agent
Instantiation
CH <objects>
CH objects
represented-by/ representsand intangible cultural heritage
hasskill

Tangible CulturalHeritage
Physical Space

2018/8/23

Physical Space

instantiated-as/ instantiation-of

Intangible
CulturalHeritage

interrelated

A Generalized Model of Digital Archives
CHDE model presented at ICADL 2017, Chulalongkorn, Nov.2017

Tangible
CulturalHeritage

26

Digital Space

Digital Space
Curated Digital
Instance

Curated Digital
Instance

member-of/ aggregates

member-of/ aggregates

Collected Set of Digital
Resources

Collected Set of Digital
Resources

DigitalCH objects

DigitalObject

DigitalObject
converted-to/ converted-fromconverted-to/ converted-fromCollected Set ofNon-Digital Resources

Collected Set ofNon-Digital Resources

aggregated-as/
related-to

DigitalCH objects

aggregated-as/
related-to

Non-DigitalObjectNon-DigitalObjectrepresented-by/ represents

Tangible
CH objects

Agent

Instantiation

represented-by/ represents
hasskill

Tangible Cultural
Heritage
Physical Space

2018/8/23

Intangible
CH <objects>

performs

Physical Space

instantiated-as/ instantiation-of

Intangible
Cultural
Heritageinterrelated

A Generalized Model of Digital Archives
CHDE model presented at ICADL 2017, Chulalongkorn, Nov.2017

Tangible
Cultural
Heritage27

Digital Space

Digital Space
Curated Digital
Instance

Curated Digital
Instance

member-of/ aggregatesmember-of/ aggregates

Collected Set of Digital
Resources

Collected Set of Digital
Resources

DigitalObjectDigitalObjectconverted-to/ converted-from

DigitalArchives

converted-to/ converted-fromCollected Set ofNon-Digital Resources

Collected Set of
Non-Digital Resources

aggregated-as/
related-to

aggregated-as/
related-to

Non-DigitalObjectMetadata
requiredfor
organizing
digitalarchives

Non-DigitalObjectrepresented-by/ represents
performs
Agent

Instantiation

represented-by/ represents
has
skill

Tangible CulturalHeritage
Physical Space

2018/8/23

Physical Space

instantiated-as/ instantiation-of

Intangible
CulturalHeritageinterrelated

A Generalized Model of Digital Archives
CHDE model presented at ICADL 2017, Chulalongkorn, Nov.2017

Tangible
CulturalHeritage

28

Very Abstracted View of Digital Archives
Digital Archives

Primary Contents
Digital Images, Sounds,
Documents

2018/8/23

Metadata
Description about- Original CH- Digital surrogates(Primary Digital Contents)- Administrative information- etc.

29

Very Abstracted View of Digital Archives
Digital Archives

Primary Contents
Metadata
Digital Images, Sounds,
Description about
Documents
Original CH is essential for digital archives
Longevity of both
Primary Contents and- Metadata- Digital surrogates(Primary Digital Contents)- Administrative information- etc.

2018/8/23

30

Sharing Digital Archives across Communities
and over Time
Digital Archives

Primary Contents
Metadata
Digital Images, Sounds,
Description aboutDocuments
Original CH is essential for digital archives
Longevity of both
Primary Contents and- Metadata- Digital surrogates(Primary Digital Contents)
Metadata Longevity- Administrative information- etc.
- Keep metadata database alive over time

- Maintain metadata schemas over time
- Maintain metadata terms over time
2018/8/23

31

Sharing Digital Archives across Communities
and over Time
Digital Archives

Primary Contents
Metadata
Digital Images, Sounds,Description about
Documents
Original CH is essential for digital archives
Longevity of both
Primary Contents and- Metadata- Digital surrogates(Primary Digital Contents)
Metadata Longevity- Administrative information- etc.
- Keep metadata database alive over time
This talk focuses on metadata- Maintain metadata schemas over time
- Maintain metadata terms over time
2018/8/23

interoperability and sharing
32

Sharing Digital Archives across Communities
and over Time
Digital Archives

Primary Contents
Metadata
Digital Images, Sounds,
Description about
Documents
Original CH is essential for digital archives
Longevity of both
Primary Contents and- Metadata- Digital surrogates(Primary Digital Contents)
Providers of digital archives are responsible
to keep bothprimary contents and- Administrative
information- etc.
metadata alive over time. However, digitalarchives are sometimes lost becauseof limitation of financial and human resources.
2018/8/23

33

Sharing Digital Archives across Communities
and over TimeDigital Archives

Primary Contents
Metadata
Digital Images, Sounds,
Description about
A simple step forward– sharing information
aboutCH
digital archives in our community
Documents- Original- Digital surrogates(Primary Digital Contents)
Approach - sharing the digital archive information,i.e., metadata about digital archives- Administrative information- etc.

2018/8/2334

Sharing Digital Archives across Communities
and over Time
Digital Archives

Metadata about
digital archives(databases)
Metadata schema may be defined by each digital archive provider but metadata schemas
should be interoperable across providers – a well-known metadata interoperability problem
2018/8/23

35

Metadata Interoperability• Conventional methods to make metadata interoperableA) All participating providers create metadata based on a standard schema
B) Create a mapping table across schemas used by participating providers• Comparison• A) allows easy integration of metadata from all providers but limited flexibility
to the providers and the providers need to get consensus on the metadata
schema as their standard• B) allows providers to create their own schemas based on their needs but
creating mapping tables is a heavy task

• In both methods, building a data model before property-level (orelement level) definition/mapping is required
2018/8/23

36

Metadata Interoperability• What is data model of metadata?
• Data model for an application problem defines entities to be described in theapplication and relationships among the entities• Data model should define classes/types of the entities• A lesson learned in a research on the Media Art Database project• Media Art Database composed of four component databases – Manga, Anime,Game and Media Arts (https://mediaarts-db.bunka.go.jp/)• Each component database schemas were initially defined separately• Rich sets of elements were defined in these schemas without explicitly
defined data models• Found difficulties in mapping their elements even though it looks that Manga,
Anime and Game are quite close
2018/8/2337

Metadata Interoperability• Metadata standards which are rather new include definition of theirdata models• Data models play important roles for achieving metadata
interoperability across different schemas (= communities)• Dublin Core Application Profiles• Generalized model for designing interoperable metadata• Separation of structural and semantic features• Structural features in three levels – Description Set, Description andStatement (+ structural constraints)• Semantic features – Semantic definition of terms should be given separatelyfrom structural features
2018/8/2338

Metadata Model for Digital Archives/Databases –Discussion for Tomorrow• A catalogue record for digital archives and databases (DA/DB)• What Entities?
• Digital Archive and Database• Agent involved in creation and/or operation/maintenance• Other entities?

• Facets for describing a DA/DB?
• Content description – title, abstract, subjects, etc.
• Stakeholders• Technological features – DBMS, user interface, API, etc.
• Rights• Other facets?
2018/8/23

39

Wrap Up
• Digital preservation is a maintenance activity of digital resources anddigital archives including their metadata• Digital preservation (or longevity of digital resources and archives)
requires metadata interoperability• Conventional element-level mapping is required for interoperating
metadata of different schemas but model-level mapping before
element-level mapping is recommended

2018/8/23

40