# Presentation: 391016

## Storage MediaLife Expectancies

**John Van Bogart**
**National Media Laboratory**
**Digital Archive Directions (DADs) Workshop**
**June 22, 1998**

**Notes:**

	Dr. John Van Bogart
	Data Preservation Scientist
	National Media Laboratory
	Building 235-1N-17, 3M Center
	St. Paul, MN   55144-1000

	Voice:	(612) 733-1918
	FAX:	(612) 575-1644
	E-mail:	jwvanbogart@nml.org

## National Media Laboratory

**Mission:****  To ensure that the U.S. Government leads the world in information superiority by working with commercial industry to catalyze, influence, and exploit emerging information technology and products that satisfy government needs.**
**NML Home Page address:**
** ****	****http://www.nml.org**

**Notes:**
The National Media Laboratory (NML) Strategic Alliance was created to bring commercial information technology expertise to the U.S. Government. Commercial expertise available to the U.S. Government through the NML Alliance includes: 
Storage and Dissemination Technology Information Management and Access Communications Technology Materials and Environment Networks Printing and Imaging 
The NML Strategic Alliance is a member of the National Technology Alliance, which has been funded by Congress since 1992 and is and currently sponsored by the National Imagery and Mapping Agency (NIMA). 
The National Media Laboratory hosted at 3M has taken a leadership role in helping the U.S. Government gain access to commercial technology since 1987. When the lab was first established, it was a unique approach to assist government reinvent the way it works with the commercial world. A streamlined approach has become more and more critical as commercially developed technology capabilities begin to surpass those of expensive, stand-alone systems developed and produced specifically by or for government. 
Initially focused on supporting advanced government requirements for data storage (specifically, magnetic tape in those days), it became apparent to the Lab that a single focus on media was not the most beneficial to its government customer. Media, after all, is only one part of the information system. 
Today, as part of the National Technology Alliance, the National Media Lab represents a strategic alliance of leading commercial companies to bring industry's best technology to each aspect of the government information system. Current NML alliance members include 3M, Ampex, General Dynamics, Imation, Lucent Technologies, Motorola, and Space Imaging. 

## NML Media Stability Studies

**Since 1989**
**To provide solutions to U.S. Government data storage system issues.**
**Technology assessment**
**System reliability studies**
**Media stability studies**
**To answer the question: “How long will my data files be accessible?”**

**Notes:**
The National Media Laboratory (NML) is an industry resource supporting the U.S. Government in the evaluation, development, and deployment of advanced storage media and systems. NML endeavors to provide a broad perspective of current progress in information technology issues, both from a commercial and a government perspective. 
NML accomplishes its mission by functioning as a: 
CONSULTANT to the U.S. Government specializing in data storage and access requirements, assessing evolving technologies, determining applicability of currently available technologies to recording programs, and influencing commercial recording systems standards to accommodate Government requirements. 
RESEARCH AND DEVELOPMENT FACILITY, which includes technology assessment and evaluation capabilities, measurements and testing laboratories, and prototype product development capabilities. This research facility is distributed across industry, university, and Government laboratories. 
TECHNICAL SERVICE ORGANIZATION providing user support at operational sites, field monitoring, rapid crisis response; and the transfer of technology and information. 
ADMINISTRATIVE AGENT in programs that require both broadly based industrial and academic resources and coordination across Government /industry boundaries. 

## Disposition/LE Chart

***Disposition/LE Chart***

- For storage at 20°C (68°F) & 40% RH

**Notes:**
Disposition charts were developed by the National Media Laboratory in 1994 to support a request from the Department of Defense.  The chart above was designed to be used in conjunction with document retention (disposition) schedules to determine which media types were acceptable for storing documents for the required length of time.
The chart above considers magnetic tape, optical disk, paper, and film media types.  In determining the LE numbers, all of the most likely modes of failure have been considered.  For magnetic tape, binder hydrolysis, pigment deterioration, and backing instabilities have all been considered.
Media quality can vary appreciably from vendor to vendor.  Also vendors have good manufacturing days and bad ones.  Therefore, a range of life expectancies is provided (shown in yellow) to consider these variation in media quality.
On the chart above, areas in green indicate that ALL media should be able to survive reliably under the storage conditions indicated.  Areas in yellow indicate that only the “best” vendors will survive for the times indicated.  Areas in red indicate that even the “best” media may be unable to survive for the times indicated.
For example, under storage conditions of 20 C and 40% relative humidity, any brand of 3480 tape should reliably store information for 5 years.  However, only the highest quality 3480 tapes can be trusted to store information for 20 years.

## Media Stability Studies

**Aging**
**	****Failure Analysis**
**	****	****Modeling**
**	****	****	****Extrapolation**

**Notes:**
At the National Media Laboratory, a variety of magnetic tape and optical disk media types have been aged in accelerated environments.  Subsequent failure analysis and modeling of deteriorating media properties allow the longevity of these media to be estimated.
A basic accelerated methodology includes:
Use of several accelerated temperature/humidity/pollutant environments to artificially age the media
The measurement of media physical properties and error/dropout rates at regular intervals.
The modeling of properties versus time using appropriate kinetic models.
Determination of the Life Expectancy, commonly referred to the LE value for a particular medium involves:
Extrapolation of models determined from accelerated aging studies using standard methods.
Application of an appropriate set of End-of-Life criteria—consisting of a material property and a value at which failure of the tape system occurs.  An example of an end of life criteria is a Block Error Rate of 220 commonly used to evaluate the longevity of CD-ROM media.

## Accelerated Aging - Tape

**Notes:**
The above chart indicates that a permanent data loss occurred as the result of mis-tracking after a D1 (digital video) tape was stored at 60 C (140 F) for 24 hours.  Storage at this temperature allowed the tape backing to undergo dimensional changes which allowed stresses in the backing to relax (stress relaxation).  The dimensional changes were great enough to distort the data tracks and raise the uncorrected bit error rate above the manufacturer’s limit resulting in hard errors.

No tape or disc is ever error free.  Error-free media would be too expensive to produce.  Instead, tape and disc systems use error correction code (ECC) schemes that allow bit errors to be corrected when the tape or disc is read. (A tape with an uncorrected bit error rate (BER) of 1 in 10,000 read can have a corrected BER of 0.)  If data is missing on one portion of the media, it may be able to be re-constructed from the redundant information written in the ECC table.
However, there is a limit to how many errors the ECC schemes can handle before the system is unable to correct the error and the user actually experiences a read error.  In the example above, the maximum uncorrected bit error rate is 1 bad bit in 1,000 bits read.  For CD-ROM discs, the maximum allowed block error rate is 220 (average).

## Media Lifetimes

**10**

**20**

**35**

**?**

**?**

**?**

**?**

**?**

***How long will***
***my tapes last?***

**27 years**

**Notes:**
Dependable storage strategy requires a knowledge of media longevities.
The prevalence of media vendors, hardware suppliers, and tape formulations make it impossible to come up with a single LE number that is valid for a given media at a specified storage condition.  Furthermore, media may fail for reasons other than those considered when determining the indicated LE values. 
It  is best to use the LE’s as “guidelines” rather than numbers “carved in stone.”  Currently, there are no accepted standards for determining the LE (life expectancy) of magnetic tape media.  Standard methods for determining the life expectancy of optical disks have been developed by ANSI, but not all manufacturers perform the necessary product testing.
With moderate care, most magnetic tapes used for digital data storage will last for 10 years. With special storage and handling, digital magnetic tape formats can reliably store information for 30 years or more. The longevity of magnetic tapes can be significantly increased by storing the tapes at lower temperatures and humidities.
Optical disc media can last for several decades. Testing by Imation/3M indicates that their CD-ROM media will last for over 100 years. Research by Kodak shows that their CD-Recordable media is also estimated to last for over 100 years. On the other hand, magneto-optical (M-O) and phase change (PD) media have a life expectancy comparable to magnetic tape of 10 - 30 years.

## ...your LE may vary...

**The quality with which the media was manufactured -- manufacturers and lots vary in quality**
**The storage temperature and humidity**
**The cleanliness of the storage environment**
**The care with which the media is handled**
**The number of times the media is accessed over its lifetime**
**The quality of the recorder used to read/write to the media**
**Technological obsolescence**

**Notes:**
Keep in mind that LE values are similar to “mile per gallon” ratings for automobiles -- “your actual mileage may vary” depending on the number of times the tape or disc is accessed during its life, the quality of the conditions in which it is stored, and the care with which it is handled/shipped.

With regard to technological obsolescence, digital tape and disk media require special hardware in order to be read.  Even if the media survives for 50 years or more, the technology needed to read the data may not survive for more than 10 or 20 years.  Thus, the life of a digital archive may be limited by obsolescence of the drive hardware rather than the stability of the media.
Technology WILL change --  Migration to new recording systems and transcription of records to new media is inevitable to preserve the records.  Some US government agencies transcribe all of their data to a proven “state-of-the-art” format every ten years.  The old records are maintained until the next transcription occurs.  In this way, the media/system is only required to last 20 years.  Furthermore, there are always two copies of the information available. 

## CD-ROM: Durability

**Source****:  ****Bill Murray, “CD-ROM Archivability,” NML Bits, 2(2), 4 (1992)**

**Notes:**
The chart shows that there is also a wide variability in the stability of CD-ROM disks produced by various manufacturers.
On this chart, the percent increase in block error rates (BLER’s) is plotted as a function of time. The CD-ROMs were aged in an environment of 60 C and 85% RH.  Some disks were unplayable after less than 100 hours in this environment.  Other manufacturers products lasted over 3000 hours.

## LE - Qualifying

** ****To completely qualify a LE value:**
**Temperature**
**Humidity**
**Failure Criterion**
**Property & Value**
**Percent Surviving**
**Confidence Interval**
**The same set of data can give many different LE values ! !**

**Notes:**
Several criteria are required to fully qualify a Life Expectancy value.
The same set of experimental data can provide different LE values depending on the end-of-life criteria used.   In his book, Optical Recording:  A Technical Overview, Alan Marchant states that “one company has variously quoted lifetimes of thirty, fifty, and one hundred years for a write-once optical disk product, basing these different estimates on exactly the same test data.”  When reviewing manufacturers’ media archival life values, it is important to understand the method and criteria by which the LE values were determined. 
The “weak link” in a data storage product will determine its lifetime.  A tape with a 80 year expected life based on magnetic property performance may have an actual life of only 30 years by reason of poor binder stability.

## AES/ANSI IT9-5 JTC

**Write standards for estimates of media longevity**
**Write standards for storage, handling, and transfer**
**Meets twice a year (June & Dec.)**
**Representatives from industry, government, and academia**
**IBM   Imation   Kodak   Quantegy **
**NARA      NIST     Library of Congress**

**Notes:**
Belfer Audio Archive, Syracuse University
CD-Info Company
Digital Audio Disc Corporation
Eastman School of Music
FBI
IBM
Image Permanence Institute
Imation
Kodak
Library of Congress
Media Machines International
National Archives

## ANSI LE (Existing)

**Life Expectancy of Compact Discs (CD-ROM) (IT 9.21)**
**Polyester Base Magnetic Tape - Storage (IT 9.23)**
**Life Expectancy of Magneto-Optic (MO) Disks (IT 9.26)**

## ANSI:  LE of CD-ROM

**Failure:****	****10 second average of the block error rate  (BLER, ISO/IEC 10149)  >=  220**
**Eyring acceleration model**
**Weibull failure distribution model**
**Report LE:  “At a storage condition of 25° C and 50% RH, 95% of the product evaluated will last __ years, with 95% confidence.”**

**Notes:**
The ANSI/NAPM IT9.21-1996 American National Standard for “Life Expectancy of Compact Discs (CD-ROM)--Method for Estimating, Based on Effects of Temperature and Relative Humidity” outlines a method for estimating the longevity of a CD-ROM disc by aging discs to failure.  The point of failure is an average block error rate (BLER) of 220.  An Eyring kinetic model is used to determine acceleration factors at each of five aging conditions.  The distribution of failure times is then analyzed as a Weibull failure distribution.  Results are extrapolated to 25° C and 50% RH to estimate a life expectancy for the sample population at those conditions.
The test requires a minimum of 80 test samples and five temperature-humidity conditions.  Discs are aged for a period of time, removed from the test chambers, BLER rates are measured, and any unfailed discs are returned to the chambers for another incubation period.  The test requires a minimum of 6-months to complete.
The life expectancy of the CD-ROM media is reported using the following language:  “At a storage condition of 25° C and 50% RH, 95% of the product evaluated will last X years, with 95% confidence.”

## LE: High Grade VHS Tape

***LE: High Grade VHS Tape***

**Notes:**
Modeling the hydrolysis of tape binder and application of an appropriate set of failure criteria allow the LE’s for a high grade VHS to be estimated at various storage conditions and temperatures in the chart above.  For the sake of discussion, it is assumed that the LE of this particular tape is 10 years when stored at 72 F & 50% RH.  Furthermore, only the detrimental effects associated with hydrolysis of the binder are considered in determining LE values.
Cooling tape results in the slowing of the chemical processes which are degrading the tape.  Drier conditions reduce the amount of water absorbed by the tape and also reduce the rate and ultimate extent of binder hydrolysis.
One point that this chart makes is that slightly higher humidities can be tolerated at lower storage temperatures for the same level of binder degradation. It also shows the benefits of lower temperature and humidity storage.
The chart also demonstrates that the life expectancy of magnetic tape can be significantly increased by slightly lowering the temperature and humidity of storage.  Life expectancies of 10, 17, and 28 years are estimated for storage conditions of 72 F & 50% RH, 68 F and 40% RH, and 65 F & 30% RH, respectively.  In these examples, both the humidity and temperature have been reduced simultaneously.  However, it is not necessary to reduce both to see a benefit in media longevity.  Either the temperature or humidity can be reduced with benefit to the media.

## Storage Conditions

**Past**
**70° F (21° C) & 50% RH (SMPTE - 1982)**
**65° F (18° C) & 40% RH (NARA-1990)**
**Future (ANSI, SMPTE, . . . )**
**For easy access:**
**60 - 74° F (15 - 23° C)**
**30 - 55% RH**
**Extended term storage:**
**As low as 50° F (10° C)**
**As low as 20% RH**

**Notes:**
Current industry standards recommend that materials be stored around 20 C and 40 - 50% relative humidity.  They may not be the best conditions for the long-term storage of magnetic tape and optical disk media.
In the case of short-lived information, storage conditions can be at or near the room ambient conditions. No special storage facilities would be required, assuming that temperatures stayed between 20 - 24 C year round and the humidity never exceeded 55% RH.
For the indefinite storage of information, special storage facilities would be required to maximize the lifetime of the media. Storage temperatures as low as 40 F and humidities as low at 20% RH are being considered.
No medium lasts forever, so transcription of information from old, deteriorating media to new media would eventually be required; however, storage conditions can be optimized to preserve the current media copy of the information for as long as possible.

## ANSI (In Draft)

**In Draft**
**LE of CD-R**
**LE of Magnetic Tape**
**Handling of Magnetic Tape**
**Storage of Optical Disc (IT 9.25)**
**Proposed**
**LE of DVD**
**Light Stability of CD-R**

## Conclusions

**To preserve digital information**
**Knowledge of media longevity is essential**
**Transcription is inevitable**
**ANSI/AES JTC**
**Standard LE methods **
**Storage/Handling guidelines**
**Future needs**
**Use of ANSI test methods -- publish LE ratings for media**
**Systems that monitor bit errors and warn of imminent media/drive failures**

**Notes:**
In order to preserver information stored on digital media in perpetuity, a knowledge of digital media longevities is essential for proper archive maintenance.  However, because of rapid advances in digital data storage technologies and the fragility of digital media, transcription of information from older to newer media at regular intervals is inevitable.
The AES/ANSI joint technical commission has developed and is continuing to develop standard test methods for estimating the life expectancies (LE’s) of optical disc and magnetic tape.  This committee has also developed guidelines for the storage and handling of media that will ensure the media can be preserved for as long as possible.
In the future, manufacturers need to use these standard test methods and provide users with reliable LE estimates for media.  There is also a great need for systems that monitor error rates on media and alert the user when the media is close to failure.  This would allow sufficient time to copy the information to newer media before documents were permanently lost.  An example of this is SMART (Self-Monitoring And Reporting Technology) which is built into some newer hard drives.  This technology monitors disk drive performance and can predict drive failure before they occur.

## Accelerated Aging - Tape

**Notes:**
The above chart indicates that a permanent data loss occurred as the result of mis-tracking after a D1 (digital video) tape was stored at 60 C (140 F) for 24 hours.  Storage at this temperature allowed the tape backing to undergo dimensional changes which allowed stresses in the backing to relax (stress relaxation).  The dimensional changes were great enough to distort the data tracks and raise the uncorrected bit error rate above the manufacturer’s limit resulting in hard errors.

## CD-ROM: Durability

**Source****:  ****Bill Murray, “CD-ROM Archivability,” NML Bits, 2(2), 4 (1992)**

**Notes:**
The chart shows that there is also a wide variability in the stability of CD-ROM disks produced by various manufacturers.
On this chart, the percent increase in block error rates (BLER’s) is plotted as a function of time. The CD-ROMs were aged in an environment of 60 C and 85% RH.  Some disks were unplayable after less than 100 hours in this environment.  Other manufacturers products lasted over 3000 hours.

## Tape - what can go wrong?

_**Pigment**_** -Loss of magnetic signal**
**               ****Deterioration of particle**
**               ****Self-demagnetization**

_**Binder**_** - Polyester Polyurethane Hydrolysis**
**                   ****- "Sticky" tape phenomenon**
**                   ****- Debris/head clogs**
**              ****Lubricant Loss**

_**Substrate**_** -Dimensional Changes**
**	****	****	****       ****- Mis-tracking**

**Notes:**
There are essentially three “weak links” in tape media:  The magnetic “pigment,” the binder, and the supporting substrate.
The magnetic pigment is responsible for storing the recorded information magnetically.  If the magnetic particle degrades over time, information can be lost.
The binder is responsible for holding the magnetic particles to the substrate and facilitating tape transport via lubrication.  The binder is subject to hydrolysis in a humid environment and can loose its integrity.  A  “sticky tape” phenomenon can result.  Lubricant levels also decrease over time resulting with a tape with higher friction.  This results in greater mechanical wear and debris generation.
The tape substrate can undergo dimensional changes as ambient humidities and temperatures change.  This can result in mis-tracking and the inability of the playback system to locate data on playback.  Excessive tape pack stresses and poor quality tape packs can permanently distort the tape backing so that mistracking becomes a permanent problem.

## Binder Breakdown 

**Notes:**
The most likely cause of tape failure is deterioration of the binder polymer used in tapes.
The binders typically used in tape formulations are polyester-polyurethane based. These materials are subject to hydrolysis in a humid environment. The polyester linkages in the polymer will react with water in the environment and polymer chain scission will occur reducing the integrity of the binder system.
This results in a "sticky tape" syndrome, characterized by high tape friction and low coating integrity. The high friction can result in tape seizing and a inability of the tape to transport through the drive. The lower coating integrity can result in debris which can cause head clogs and dropouts. Temporary and permanent data loss can result from binder degradation.

## Optical Disk Structure

**Notes:**
In an optical disk system, a laser is used to read data from a rotating disk.  The data layer on the disk consists of a pit, or bump, or magnetic region which disperses, reflects, or rotates the incident beam.  Anything which interferes with or changes the optical properties of the data layer or disk substrate can result in data loss.
In general, the polymeric components of a disk can degrade.  The metallic components can corrode or de-alloy.  If the aluminum reflective layer on a CD-ROM disk corrodes, its reflective properties will change and data can be misread.
The optical clarity of the substrate is important in those systems where the laser must pass through the substrate.  CD-ROM and M-O disks fall into this category.  Anything which interferes with the transmission of the beam, such as a scratch, or reduced optical clarity of the substrate, can result in a data error.
Most of the durability problems encountered with M-O disks are associated with degradation of the magnetic layer.  The layer is subject to corrosion, which has given M-O a poor stability rating in the past.  The magnetic layers on M-O disk can be quite brittle.  Subjecting M-O disks to temperature and humidity fluctuations can result in cracking of the magnetic layer.

## For a 100 year life . . .

**Necessary to archive system along with the media**
**Sufficient supply of spares for wearable parts (heads, LED’s, drive motors)**
**Sufficient number of “new” systems to replace irreparable, worn out “old” systems**
**Application software and host to “read” the data**
**Technical expertise / service manuals**

**Notes:**
To ensure that the media will be readable far into the future, it may be necessary to archive system components along with the media.  For a 100 year life, recording systems and sufficient spare parts will need to be archived along with the data storage media.  Media with life expectancies greater than 20 years are capable of out-surviving existing recording system technologies. 
It is inevitable that technologies will advance.  Hardware manufacturers need sales in order to grow their business.  The more that they can develop and sell technology, the better off their businesses will be.
To help put this into perspective:  If I gave you a Word Perfect 1.0 document created in a DOS environment on an 8” floppy disk, would you be able to read it?  Do you have the drive hardware, the software, and the operating system necessary to do so?

## Archival Storage Technology

***Archival Storage Technology***

**Look for industry accepted standards**
**Ensures compatibility of media**
**Interchangeability with other drives**
**Be wary of proprietary designs**
**Is it a widely used technology?**
**Tends to “guarantee” the permanence of  technology**
**Equipment, parts, support readily available**
**Less expensive**
**Established Migration Path?**
**Beware of emerging technologies**
**No track record, no standards, no market**

**Notes:**
When choosing a digital storage technology for archival purposes, the permanence of the technology should be an important consideration.
Industry standards ensure compatibility which also ensures longevity.  Data recording systems are more marketable if the buyer can be assured of interchangeability of media with other drives.  A proprietary design is only as reliable as the company behind it.  If that company goes out of business, that design will be obsoleted. 
A technology which is widely used in the consumer marketplace should have a very long product life.  The consumer acts as a buffer for technological change.  Even though technologies may advance, the consumer won’t necessarily rush out to buy it. This ensures a longer life for the older technology and guarantees the availability of drives and media for that technology. 
Be cautious of emerging technologies.  They do not have the track record of the older technologies and therefore may not have all of the “bugs” worked out of their system.  It is also difficult to assess the life expectancy of a particular medium in a system when there hasn’t been a lot of practical experience with the system.  Emerging technologies typically have proprietary designs and no standards.  In today’s volatile market, emerging technologies are not very stable and could disappear “overnight” leaving the user with an obsolete technology.

## Change in Magnetic Remanence at 75C - 90% RH

**WEEKS**

- -10

- 0

- -20

- -30

- -40

- -50

- -60

- -70

- -80

- 1

- 3

- 5

- 7

- 9

- 11

- 13

- 15

- 0

- 2

- 4

- 6

- 8

- 10

- 12

- 14

- 16

- NML ßCam SP Model

- D2-a

- D2-b

- D2-c (4/91)

- D2-d

- D2-e

- D2-f

- D2-c (10/91)

**Notes:**
Media vendors vary in quality.  This plot demonstrates this fact. This plot shows the deterioration in magnetic remanence for different brands of D2 digital videotapes when aged at 75 C & 90% RH.  Note that vendors have varying levels of stability.
Differences can also be observed between different lots of tape from the same manufacturer.  There are two curves on the chart for Vendor “C” purchased at two different times.  One lot degrades at twice the rate of the other lot.

## Digital Media vs. Film & Paper

**Digital media is not as stable**
**Lower practical lifetimes (10 - 30 years)**
**Catastrophic failures**
**Difficult to assess “health” of the storage medium**
**Transcription required at regular intervals to preserve information**
**Rapid advances in technology**
**Prevalence of several formats/ types**

**Notes:**
As an information storage medium, magnetic tape is not as stable as film or paper. Properly cared for, film and non-acidic paper can last for centuries, whereas magnetic tape will only last a few decades. Use of magnetic media for storage is further confounded by the prevalence of several formats, media types, and by rapid advances in media technology.
Books have virtually maintained the same format for centuries, have almost exclusively used ink on paper as the information storage medium, and require no special technology to access the recorded information.
Likewise, newer microfilm, microfiche, and movie film are known for their stability when kept in proper environments, and viewing formats have not changed significantly over the years.  New film records have an excellent resolution and permanence which is unsurpassed electronically without requiring huge storage volumes. 
Keep in mind that by moving from paper/film to digital storage, you are moving to an inherently more unstable medium.  Except in cases of extreme deterioration of the original, the film/paper record will most likely outlive the digital storage medium. 
To maintain an electronic record “permanently,” transcription from older, deteriorating media to new media will be required at regular intervals.