# Presentation: 004618

## Prospects and Challenges of Migrating to Digital Moving Image Archives

**R. Justin Davila**
**Co-Inventor, SAMMA System**
**and Consultant,**
**Media Matters LLC**

**Notes:**
Good morning. My name is Justin Dávila, Co-Inventor of the SAMMA System, Media Matters, LLC. I am here to talk about two major projects Media Matters has undertaken, besides the SAMMA project Joanne just described. The first is an investigation of uncompressed and lossless compression for moving image preservation performed for the Dance Heritage Coalition. Experiments included MXF and AAF container file types with a variety of preservation codec candidates.  Promising results with Motion JPEG2000 led to creation of an encoder board, which will be demonstrated live.

## Focus on Motion JPEG2000

- Past
- Digital Video Preservation Format
- Present
- Real-Time Lossless-Compressed Audio-Video Encoding Prototype
- Future
- From Prototype to Production

**Notes:**
This presentation will focus on our work with Motion JPEG2k as a preservation format. We will cover the past, how we made the decision to create the first real-time hardware audio and video solution based on MJEPG2k, then move on to the demo of our working prototype, and talk about some future directions with the card as we are moving to the production version.

## PAST: In Search of Real-Time Lossless Compression

## Goals of SAMMA

- System for the Automated Migration of Media Assets
- Large Scale Media Migration Projects
- Higher Productivity
- Lower Cost
- Higher Quality

**Notes:**
I’m glad that we’re able to present our demo of our card after Joanne’s presentation on her experience implementing the Media Matters System for the Automated Migration of Media Assets at Yale University’s Fortunoff Collection of Holocaust Testimonials. The principles that guided our development of an automated robotic migration system are the same ones that led us to Motion JPEG2000. And those principles were to crate a comprehensive solution for large audiovisual collections that would deliver three things: much higher productivity, much lower cost, and much higher quality in the migration process.

## Missing Links

*Custom* Tape Inspector-Cleaner: Physical Tape
*Custom* TBC: Analog Signal
- Archive File Format: Preservation and Access

**Notes:**
There were three “missing links” in this process that we had to build custom for this project. The first is a custom tape inspector-cleaner, a tool that would identify major physical problems with tapes at the same time that it would clean the tape, all the while reporting tape defects back to the system as metadata. Then we need a custom time base corrector/frame synchronizer, which would correct the analog RF signal itself, and also report problems with the signal back to the system as metadata. We created both of these tools from scratch, the first is SAMMA-Clean, and the second in SAMMA-Sync. Finally, we needed a processional archive file format that would provide for both the preservation and access of the content of the collection. And this is what we are demonstrating today.

## Digital Video Preservation Reformatting Project

- Researched for the Dance Heritage CoalitionElizabeth Aldridge	Executive Director
- Jim Lindner, 	Principal Investigator
- Justin Dávila	Senior Researcher

**Notes:**
We first researched Motion JPEG2000 as part of a consulting engagement with the Washington DC based Dance Heritage Coalition, which is headed by Elizabeth Aldridge. Jim Lindner, who many of you know from Media Matters and also his previous company VidiPax, was the principal investigator, and I served as senior researcher on the project.
Dance is a unique performing art in that is so ephemeral. Much of the history of dance in America is on video tapes, and is in danger of being lost due to the typical forces endangering all magnetic media archives: deck obsolescence, the cost of maintaining tape facilities, and the cost of migrating the sheer number of tapes to a more permanent format. 

## Digital Format Requirements

- Quality
- Usability
- Preservation

**Notes:**
The basic directive of the project was to find a digital file format that would meet the three goals of the project. Quality, Usability, and Preservation.

## Digital Format Requirements

- Quality
- Measurable Factors
- Resolution
- Luminanace
- Chrominance
- Loss of Data in Compression

**Notes:**
For quality we wanted to make sure we could effectively, or better yet identically, replicate all the measurable factors of the audiovisual source. Some examples are resolution, luminance, chrominance. We wanted to make sure we lost as little data as possible in the case of compression.  

## Digital Format Requirements

- Usability
- Editable (Production-Friendly)
- Stills/Freeze Frame
- Metadata (Information about the Item)
- High Resolution (HDTV)

**Notes:**
For usability, we wanted to make sure the file was not just preserved, it was preserved in a way that would be useful in a production environment, far into the future. We needed to make sure it was editable, preferably with no interframe compression This also means each frame can be selected as a still or freeze frame, which is a generally useful feature, but really essential in dance. It had to accommodate extensive metadata of all varieties: descriptive, digital provenance, and technical. It had to support future uses at a good quality, including High Definition or digital film applications. 

## Digital Format Requirements

- Preservation
- Long Term Support
- Permanent Access
- No Proprietary Equipment
- Open Standard

**Notes:**
Finally, we had to make sure our selection served the preservation needs for the content. We had to ensure long term support for the standard, and not require any proprietary equipment to access the content in the future. This means an open standard would be an attractive candidate.

## All Paths Lead to Motion JPEG2000

- Lossless Compression
- No Interframe Compression
- Access File Scalability
- Open Standard

**Notes:**
Motion JPEG2000 uses a special compression based on new wavelet-based algorithms, which allow for mathematically lossless compression. There is no loss of information in the process. What goes in is what comes out. Think of it as a Zip file, with compression ratios of around 3:1. Further, it met our requirements for no interframe compression, meaning the every frame of the file is useful as a still frame or key frame. Another trait of Motion JPEG2000 is the fact that lower quality or lower bitrate files can be “peeled off” the original file for access copies – we call this access file scalability. Finally, it’s an open, well published standard.  

## File Wrappers

- Materials eXchange Format (MXF)
- Production-Friendly
- Defines data structure for audio and video material (*essence*) plus associated descriptive and structural Metadata
- Workable subset of larger AAF wrapper format

**Notes:**
One other conclusion in our report was on file wrappers. File wrappers encompass the basic audio-video essence as a package, usually with additional information such as for a production environment. This was very important to accommodate future production uses of the archived content. Our recommendation was for MXF, the Materials eXchange Format, developed by the Pro-MPEG group, and being standardized by SMPTE.

## PRESENT: Demo of Motion JPEG2000 Real-Time Audio-Video Encoder Prototype

**Notes:**
Now,on to the “present”, our Motion JPEG2000 audio-video encoding card.

## Prototype Card

- Prototype useful for demonstrating internally and to select customers.
- Production card being farmed out to board manufacturers.

**Notes:**
What we have here is our prototype board, straight from our lab. We thought it would be better to demonstrate the capabilities of the existing prototype, even though it lacks some functions of our final production card. The final production cards are being farmed out to manufacturers now.

## Features

- Uses Analog Devices ADV202 JPEG2000 Compression Chips
- Encodes Standard SDI component video into compressed JPEG2000 Stream. Encodes AES digital audio into standard BWF file.
- Decodes using a second chip for pass through into standard 525i/625i component video and AES audio out.

**Notes:**
The major features of the card are the chips that do the “heavy lifting” of real-time Motion JPEG2000 compression and decompression, the Analog Devices ADV202 chipset. It takes standard 525/625i SDI signal in, plus AES audio, and sends out standard 525/625i SDI plus AES audio. 

## Demo Limitations

- Vertical Blanking in SDI
- ADV202 requires long blanking for 525i
- Audio Sync
- Constant A/V sync delay (approx .25 sec) requires adjustment
- Both issues exist only in this prototype board and will be resolved in production model

**Notes:**
Here are some limitations on this prototype that will not exist in the production run. With this card, we have an issue with a setting in the SDI stream that controls vertical blanking, which will explain a few lines of grey at the bottom of the decoded playback. Also, we’ve got a slight delay on the video compared to the audio, just a fraction of a second. We’ve already got fixes in place for the production runs. These issues are only for the prototype.

## DEMO

## Slide 18

## Slide 19

**Notes:**
Here’s a quick schematic of our set-up today. Here’s the Motion JPEG2000 encoding card housed in our demo box, which is located over there. This is the kind of application of the card that we expect to sell as a standalone Motion JPEG2000 encoding station, where video goes in as ADI and files are made and stored locally, temporarily, and then SDI of those files goes out. Here we also have our time base corrector “SAMMA-Sync” which does analog to digital and digital to analog conversion for video, and a standard analog to digital and digital to analog converter for the audio.  

## Overview	

**Hardware Encode**
**ADV202(JP2) + BWF**
**Software Decode**
**BWF**
**Hardware Decode**
**ADV202(JP2) + BWF**

**Notes:**
Here’s an overview of what we’ll be demonstrating.
We’ll be showing a real-time audio-video encoding using our Motion JPEG2000 hardware to make a JP2 encoded video file and a BWF standard broadcast wave file for audio.
We will play back the encoded audio portion for you directly from the computer. The video portion requires processing to strip out frame headers specific to ADV202 chips, which takes a little time, so we’ll exclude the software video playback from the demo.
Finally, we’ll show real-time hardware decoding and playback of the audio and video files together– and this is a first!

## Slide 21

**Notes:**
Excerpt from STREB, Joyce Theater, 19 December 1997
Concept and Choreography by Elizabeth Streb
Performed by Hope Clark
Videotaped by Dennis Diamond of Video D Studios
Excerpt copied from Betacam SP
Courtesy of Dance Division, The New York Public Library for the Performing Arts.

## FUTURE: From Prototype to Production and the Holistic View

**Notes:**
Now we’ll give you an idea of the final steps between the prototype card and the production card, and describe the path we see for JPEG2000 files in archive and production environments.

## Video

**ADV202 – “raw”**
**JP2**
**J2C**

**Notes:**
We currently support the raw adv202 format, which encapsulates the legal JP2 or J2C Motion JPEG2000 file with an extra header for each frame which is used by the chip to accelerate hardware playback. We have a tool available to strip that header to get to the Jp2/J2c file underneath.

## Audio

**BWF: 4 Gigabytes**
**4 Ch @ 24bits/ch = 124 Minutes**
**2 Ch @ 24bits/ch = 248 Minutes**
**4 Ch @ 16bits/ch = 186 Minutes**
**2 Ch @ 16bits/ch = 372 Minutes**
**Extended BWF, RF64**
**No Limits**

**Notes:**
Currently there are some limitations inherent to the BWF format that have to do with internal 32bit addressing for that format, which results in a limit of how much we can record based on bitrate and number of channels, as shown here. This will be fine for most professional tape-based media used as source.

We will continue exploring audio file formats that have no limit, or no practical limits, such as proposed Extended BWF, RF64, or other options.

## Audio-Video

**JP2(or J2C).bin ****+ ****BWF.wav ****=****MJ2.avi(or MJ2.mj2c)**

**Notes:**
So here’s one of the big equations. This is how we make a portable, wrapped audio-visual file.
We take the JP2 MotionJPEG2000 file with a .bin extension and add the Broadcast Wave file with a .wav extension and turn it into a linked MJ2 MotionJPEG2000 file wrapped in a .avi (or potentially a .mj2c “container” file). Now we have an audio-video file.

## Metadata

**Media Matters Metadata (MMM.xml)****(LoC METS-Friendly)**
**Descriptive Metadata**
**Technical Metadata **
**Digital Provenance Metadata**
**From SAMMA Reporting Devices:**
**Tape Inspector-Cleaner**
**Time Base Corrector **
**Video Analyst**
**Audio Analyst**

**Notes:**
Now that we have an audio-video file, we want to attach to that all the metadata we have on this file. We have developed a schema based on the Library of Congress’s METS structure, which supports sections for categories like descriptive metadata, technical metadata, and digital provenance, which allows us to document the process of an analog item becoming digital. Our SAMMA system collects four types of live data from the migration process: metadata from the cleaner, the time base corrector, the video analyst and audio analyst. 

## Wrapped Metadata

**MJ2.avi ****+ ****MMM.xml****=****MXF.mxf**

**Notes:**
And finally, here’s the other big equation. We take the audio-video file and add the metadata and form a production-friendly, metadata-rich, lossless audio-video MXF file.

## MXF Storyboard

**Notes:**
This is an example of a storyboard view of an MXF viewer application. We can see the distinct “event tracks” that contain various metadata events that are recorded from the distinct reporting devices we have in the SAMMA system: the Cleaner, the TBC, the Audio Analyst, and the Video Analyst. We can see that MXF supports this kind of metadata track browsing, linked by time code to screenshots.                                                                                                                                         

## Questions?

- R. Justin Davila
- Media Matters LLC
- _[justin@media-matters.net](mailto:justin@media-matters.net)_
- www.media-matters.net

**Notes:**
Questions?