Born Compressed:
Should the Preserva5on Community
Embrace Lossy Video Compression?

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Content
Point of view of users of audiovisual achives:„It makes no sense to ‚inflate‘ digital bornc o m p re s s e d t o u n c o m p re s s e d / l o s s l e s s l ycompressed files.
Those files are very cumbersome to handle forresearchers who are only interested in the image
content.“No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Content

> Content(1)Technical characteristics and access hierarchyof archival files.
(2)
The use of lossy compression in archival masters.

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Technical characteristics

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Technical characteristics

Presentation
SourceArchival
StorageViewing

SIP

AIP

DIP

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Technical characteristics

Source•„What you get“•Analogue / Digital•

Any (data) format•Do not dispose of !

•Contains and comes with metadata

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Technical characteristics

Archival Storage•

Long term digital storage•

Archival file format•

Digital representation of source element•

Metainformation is processed and added

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Technical characteristics

Archival Storage
Born digital files are converted into anarchival file format which complies withthe requirements for archival fileformats, to improve its longevity.
Equal to analogue and tape source
elements, the digital born file should
be kept as received.
No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Technical characteristics

Archival Storage - Conditions„Openness“ / non-proprietary
High level of adoption
Low complexity
High level of documentation (standard)
Robustness to errors
Low dependency on specific hard- and software
No encryption

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Content

> Technical characteristics

Presentation•

High quality master element•

Any suitable file format•

Visual representation of source element•

Contains important information about
the visual representation

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Technical characteristics

Viewing•

Low quality element•

Any suitable file format•

Limited visual representation ofsource element•

Metadata helps to correctly interprete
what you see/hear

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Access hierarchy

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Access hierarchy

Presentation
SourceArchival
StorageViewing

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Access hierarchy

Search Engine

Presentation

Viewing M
GeographicallyArchival Storage

MNo Time To Wait 3, 25. & 26.10.2018 – David Pfluger

2nd CopyMSource (Digital Born)

Source (Analogue/AV Tape)

SeparatedM

> Access hierarchy

EASY

Search Engine

Presentation

Viewing M
GeographicallyArchival Storage

M

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

2nd Copy

M

Source (Digital Born)

Source (Analogue/AV Tape)

SeparatedM

> Access hierarchy

Presentation

LIMITED

Viewing M
GeographicallyArchival Storage

M

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

2nd Copy

M

Source (Digital Born)

Source (Analogue/AV Tape)

SeparatedM

> Access hierarchy

LIMITEDSearch Engine

Digital storage

Presentation

Digital archivalstorage

Archival Storage

Viewing

M

Source (Digital Born)

Archival storageof phys. carriers

Source (Analogue/AV Tape)

No Time To Wait 3, 25. & 26.10.2018 – David PflugerM> Access hierarchy

Search Engine

Presentation

RESTRICTED

Viewing M
GeographicallyArchival Storage

M

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

2nd Copy

M

Source (Digital Born)

Source (Analogue/AV Tape)

SeparatedM

> Access hierarchy

Search Engine

Presentation„NONE“Viewing M
GeographicallyArchival Storage

M

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

2nd CopyM

Source (Digital Born)

Source (Analogue/AV Tape)

SeparatedM

> Conclusion 1
Point of view of users of audiovisual achives:„It makes no sense to „blow up“ digital bornc o m p re s s e d t o u n c o m p re s s e d / l o s s l e s s l ycompressed files.
Those files are very cumbersome to handle forresearchers who are only interested in the image
content.“No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Conclusion 1
When accessing a digital AV element in an archive
researchers should not get in touch with the archival
data file format.
For viewing purposes the viewing files are made.
In almost all cases these files should suffice together
with the corresponding metadata.
Else the archive you are consulting needs to fix theiraccess hierarchy.
No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Topic in need of clarification!

Presentation
Source

SIP„Rawscan“
Graded Master
Restored Masteretc.

AIP

Viewing
DIP

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Topic in need of clarification!

Presentation
Source„Rawscan“
Graded Master
Restored Masteretc.

Viewing

M
SIP?

AIP?

DIP?

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Content(2)The use of lossy compression in archival masters.

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

Agathe Jarczyk, Reto Kromer, Yves Niederhäuser, David Pﬂuger
No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Lossy compression

Archival Storage - Conditions„Openness“ = Non-proprietary
High Level of Adoption
Low Complexity
High Level of Documentation (Standard)
Robustness to Errors
Low Dependency on Specific Hard- and Software
No EncryptionNo Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Lossy compression

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Lossy compression
Art collecting institution• Mixed collection• ca. 145 AV artworks• 135 in SD video, carrier based(U-Ma5c, Beta SP, VHS, Laserdisc, DigiBeta, DVD)• 10 digital born in HD video, lossy compression• In-house production of documentation in HD video

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Lossy compression

Assuption for the in-house production of documentation:
30% of works are documented with a 60 min HD video.

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Lossy compression
500 TB

Uncompressed
ffV1

400 TBXDCam HD
300 TB

200 TB

100 TB

> Lossy compression
Drawbacks• Chances that the data is lost is higher for thedocumenting footage.
• More efforts need to be invested in regular checks
for obsolete formats.
• You may have to transcode in a future migration stepwhich may decrase the image quality.

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Conclusion?

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger

> Thanks for your attention!

david.pfluger@saeure.ch

No Time To Wait 3, 25. & 26.10.2018 – David Pfluger