by hilbert42 | 2025-09-20 18:14:54+00:00
"…you want them to be archived now when they're readily available not years from now when they're hard or impossible to find."

Exactly. Over the centuries humans have used various techniques to record information they've wanted to keep for posterity and almost every newer innovation they've adopted has resulted in information being stored for a shorter time than the older technology that preceded it.

That's a sweeping and overly general comment but I'll justify it with several examples from history. First, there are many reasons why people would want to replace a time-honored way of documenting information by using alternative methods. My purpose here to draw attention to how with each new technical innovation storage longevity ends up being shorter than the previous generation.

Examples:

1. Messages chiseled on stone has a longevity of many millennia. Examples: Egyptian hieroglyphics, the Rosetta Stone.

2. Information written on parchment and vellum can, with reasonable care, be in good condition after 1000 years or more. Examples include the Doomsday Book, Magna Carta and the Lindisfarne Gospels (it's ~1300 years old).

3. Paper has the advantage of higher storage density especially if it's the substrate for printed text. Longevity can be high for high quality paper, flax types etc can last over 500 years, whereas cheap paper as in paperbacks and newsprint has a much shorter lifespan, likely 50 - 75 years at most.

4. Painting (oil-on-canvas) has, with care, good longevity, at least 500-plus years. Works of the great masters, Caravaggio, da Vinci (Mona Lisa), et al, are nowadays still in reasonable to good condition.

5. Photography has several advantages over art, it's capable of rendering an image accurately and it's much easier to reproduce multiple images than drawing them all by hand. Longevity and quality of photographic images depends on the technology that's employed. Here are several examples.

-- Wet-collodion process (negative on glass plate). Introduced 1851. Advantage: high resolution, disadvantage: not orthochromatic/panchromatic, sensitive only to blue light. Longevity: if stored with care images will last 200-plus years. Examples Civil War photos of Mathew Brady, Alexander Gardner and others.

-- Kodachrome color transparency film. Introduced 1935. Advantage high resolution, high color accuracy. Longevity: ~200 years under ideal storage conditions.

-- Kodak Kodacolor negative (late 1940s to ~1960). Longevity: only a few years, image fades even when stored under ideal conditions. This tech was an unmitigated disaster, many families lost treasured wedding photos etc. because of fading.

-- Eastman color print film (theater release stock). Subject to considerable fading when stored, film cannot be used for archiving, it's useless if stored for several decades.

-- B&W film stock on acetate base. Longevity: 200-plus years under proper storage conditions.

6. Sound Recordings:

-- 78-RPM shellac disks. ca 1905 - ~1955 (cylinders were earlier, first produced in late 1880s). Sound quality poor to fair. Longevity: if stored under ideal conditions and handled with care then lifetime is >200 years (there are good, well kept 78 recordings that are now about 120 years old and they're still in excellent condition).

Note: that figure is the archive lifetime. Unfortunately, the original instruments—Edison phonographs and like—that 78s were designed to be played on cause considerable wear and damage to 78 RPM disks.

I cringe every time I see YouTube videos of well-meaning owners of treasured acoustic gramophones playing 78 recordings on them. They seem completely oblivious to the damage they're doing to their recordings.

(Just because 78s were designed to be played on these players it doesn't mean they're not damaged by them. In those days pickups were without electrical amplification and to get maximum sound level considerable weight was applied to the stylus. Moreover, to improve stylus (steel needle) tracking an abrasive was added to the shellac which abraded the stylus to best fit the recorded groove. It worked both ways, both the record and stylus were worn during playback).

-- Vinyl LP High Fidelity recordings. 1948 to ~1980 (now in limited revival). Longevity: ~200 years or perhaps longer for a recording kept for archival purposes..

7. Magnetic Recordings—Audiotape, Videotape and Hard Disks. ca 1940s to present—athough its use has declined in recent years. Magnetic tape was often used to make master recordings for both vinyl records, videos (VHS etc.) and in television production, and it's still used for data storage, QIC cartridges, etc. Also, magnetic recording is still the underpinning technology in rotary hard disks.

Longevity: That said, magnetic tape and similar media, hard disks, etc. suffer from loss of magnetic remanence. Simply, the magnetic intensity on storage media decays over time, thus so does the recorded information.

Many factors contribute to the decay of information on magnetic media (which I cannot cover here) but in comparison with the older media (e.g, vinyl, 78s) its lifespan is almost pathetically short. One risks one's data if one uses a hard disk much past its short lifespan of about five years. Moreover, just archiving one's data on a hard disk and assuming it'll be OK because the drive is not actually being used is a risky business. The only way to guarantee the integrity of one's data is to copy it—rewrite it to media before its remanence falls below the threshold where data cannot be recovered, ideally this should be done within the drive's specified lifetime.

With analog recordings loss of remanence shows up as loss of signal-to-noise ratio. I've personally known people who've dug out VHS videos of their wedding to show to their kids on their 21st birthday only to find the tapes not viewable.

8. NAND Memory. These days just about everyone has moved to NAND flash memory because of its speed and convenience. It's hard to buy a PC without either an SSD or NVMe storage, and everyone uses USB trumb drives. NAND storage is great stuff, we all love it.

However, if one is not careful and proactive NAND is a time bomb waiting to destroy your data. First, NAND deteriorates and wears out with usage, second—like magnetic remanence—its electronic storage decays with age even when it's only being used for archival storage.

I speak from experience here, some years ago I created an archival backup of all my important photos and stored them on a new 500GB SSD which I put in a safe place for storage. Several years later when I checked the drive it was completely unresponsive and I thought it dead. As luck would have it, about a half hour later the SSD eventually came back to life.

What happened was the drive's controller locked drive whilst it refreshed its floating-gate charges. Luckily, charges had not decayed below the threshold where my data would have been irrecoverable. If say I'd not checked the drive for another year then chances are that I'd not have been so lucky. (That's just one incident, I'll refrain from discussing other recent NAND failures.)

Yes, it was a backup and my working copies were OK, but the point is obvious, one cannot put modern NAND storage aside and simply just forget about it indefinitely and hope one's data will still be viable. Unfortunately, nothing could be further from the truth.

We shouldn't be surprised by what happened here when we care to look at how NAND flash actually works. Frankly, it's amazing that it works at all let alone the fact that it works so well, NAND is truly a masterpiece of modern semiconductor fabrication.

NAND's functionality depends on quantum tunneling and that we use the fact to force a few hundred electrons across an 'insulated' barrier where (hopefully) they'll stay stored in the FET's floating gate for an indefinite length of time.

What I find amazing is not that quantum tunneling works and that we can store electrons in a floating gate or charge trap (and that's amazing enough of itself) but that the insulation is so good (stray resistance is so high) that it can take some years for this almost infinitesimal charge to leak away and dissipate.

Whilst we can view quantum tunneling and charge storage as a state of stable equilibrium (in that in an ideal transistor the charge would be stored indefinitely), that cannot be said for a real-world device, eventually the charge will dissipate and with it goes your data. NAND storage has many advantages but in comparison with other storage tech its both ephemeral and short lived.

We have a world now running on NAND technology and yet we've no immediate technology in the wings that'd be a better replacement—one that's intrinsically or inherently stable by design, and that's a pretty unsatisfactory and unnerving situation. Moreover, the situation is made worse by our blasé attitude and headlong rush to adopt multilayered 3D NAND which further reduces the electron count in a floating gate. By design, we're deliberately making NAND more unreliable because of our demand for even more storage.

When one thinks about it, it's pretty outrageous that the world is having to rely on dozens of data centers so as to achieve some degree or guarantee of longterm data reliability.

It's hard enough now to find and keep track of information without revengeful moneygrubbing bastards screwing the Internet Archive. We need every bit of reliable archival storage now and their actions are selfish and counterproductive. If this nonsense continues then heaven knows what the future holds for longterm data storage.