Write once, read forever, backup media advice - Page 1
Forum section: electronics / general

author: Infraviolet
author_rank: Super Contributor
timestamp: August 17, 2024, 06:47:43 pm
content: Before I start, yes I know the best ways to manage backups involve constant attention and routines involving checking the checksums at a defined interval and constant migration of data on to new media but...And before format obsolesence concerns come up, the size of the backups in question is as big as it is because for any files in a specialised format I've also included fully offline installer files necessary so as to install the programs that can read the files on to any reasonably normal Linux distro circa 2023.I'd be really interested in an last resort emergency backup which wouldn't need as much attention.My parameters are:..20GB (smallest case) to 100GB (largest case) size, 20 gigabyes of the most crucial stuff, but if there's a cheap way I'd consider this method for a further 80 gigabytes of relatively important stuff..Write once to the whole medium, if there's free space left over I'm ok with it being "wasted" and never being able to write to it again in a future session..Read forever, but I would be ok with something which one could only read perhaps 10 or 20 times so long as those 10 or 20 times could be separated across many decades of future time (so something you might read in 50 years time, and then again a further 20 years later, then after that second read use the copied data to make a new copy)..Stays unpowered, entirely, and offline except during the initial writing and during subsequent readings..No reliance on a cloud service, or other continually paid service, to host it. I want something I can rely on, rather than having to rely on a company that may go bust any day.Hard-drives and flash memory are subject to bit rot.Normal optical disks are considered 50 years life at the very best, with laser burnt dyes decaying by then. Stamped commercially mass-produced discs have much longer life expectancies, but there's no apparent way to make stamped discs yourself.M-disc looked a good idea, but doesn't seem to be made any more in its most resilient DVD format, and again it's hard to find a burning method one can be sure of as Linux compatible. I hear that since M-disc blu-rays came out, the only type of M-disc still being manufactured, they've been barely more resilient than regular laser-written blu-rays, and again I'm not sure if good Linux compatiible blu-ray burners exist.People often joke about paper or engraved stone as long term means of storage, but for multiple gigabyes this is a non-starter.I do note that EEPROM type memory often is rated for 100 years retention, as is the type of memory used to store program data in microcontrollers (AVR and such datasheets imply as much). But the idea of building a multi-eeprom chip circuit for this job collapses when you find the biggest eeprom chips on sale are in the few megabytes (or few megabits where the filters by size on seller websites aren't so good) range, so assembling 10s of GB of these would be huge and very pricey. If there were eeproms of sevral gigabytes each these would seem suitable.What is likely to be my best option then?Thanks

author: ebastler
author_rank: Super Contributor
timestamp: August 17, 2024, 07:28:01 pm
content: If you are targeting "many decades", you will also have to think about availability of suitable reader devices and interfaces to connect those to. (Beyond the media lifetime which you discussed.)I'm afraid it is not a realistic goal. Be prepared to check the media health every few years, and to copy to new media (and potentially a new format) at least once per decade.EDIT: Software which can still interpret the old data formats will be yet another challenge...Having said that, my 44-year-old Apple II is still working, and 95% of my old floppy disks are still readable too. Transferring the data to a modern PC is still supported via a serial transfer cable. The PC itself can't make any sense of most of the old data formats though -- but at least it runs Apple II emulators which can. So if you bet on the right horse (i.e. host system and media) you might be alright. But it's a bit of a gamble...

author: IanB
author_rank: Super Contributor
timestamp: August 17, 2024, 08:21:53 pm
content: Your question is unrealistic.For 20-30 years time, use a standard external USB hard drive. It is highly likely there will be USB support continued in the coming decades. USB devices from 20 years ago still work. Magnetic storage is quite stable, as long as you store it carefully away from magnetic fields and mechanical shock.For 70-100 years time, you are not talking backup, you are talking archival storage.One way to do this is to transfer data to microfilm using some kind of bar code or QR encoding, and store it with a detailed description of the encoding so that future archivists can construct a reader and retrieve the data.But this is not at all dense as a form of storage, and not something you can just buy and use. So it is not an answer to your presumed question.The only form of storage that has been proven to last centuries and still be recoverable is books, because it uses a stable medium and requires no technology to read. Technology changes too fast today for any form of technology to be proven durable. Hard disks and floppy disks from the 1980's can still (just) be read, but it's not clear for how much longer. This is not due to "bit rot", but due to mechanical degradation and obsolete interfaces.In summary, if you seriously want data to be accessible in 70 or more years time, you need to worry about being able to read the medium much more than preserving the medium itself.

author: soldar
author_rank: Super Contributor
timestamp: August 17, 2024, 08:51:49 pm
content: Punched paper tapehttps://en.wikipedia.org/wiki/Punched_tapeor printed code similar to qr code and optically readable.

author: rhodges
author_rank: Frequent Contributor
timestamp: August 17, 2024, 08:57:02 pm
content: QuoteMaybe movie film?QuoteOr provide the source code in COBOL?

author: daqq
author_rank: Super Contributor
timestamp: August 17, 2024, 09:03:26 pm
content: There's this:https://en.wikipedia.org/wiki/5D_optical_data_storageBut not sure if it's commercially available. On a budget, something inspired by it could possibly be done? Far lower data density "engraved" into glass?

author: Infraviolet
author_rank: Super Contributor
timestamp: August 17, 2024, 09:03:44 pm
content: Ok, so "archival" might well be a better word than "backup" but I think the integrity of the data on the medium still outweighs the difficulty of reading the medium for these circumstances. Yes if one is leaving data to distant future generations then readability becomes a problem, but for leaving it for as long as one's own lifetime plus the lifetime of at most one more generation... well nowadays it is still very possible to buy USb connectable floppy disc drives and if it wasn't there'd still be a decent hope for someone who needed to read a floppy disk on a one-off basis to pay some money to the owner of a floppy drive reader then come round for a visit and copy the floppy on to more current media. The reason that is harder for things older than floppy drives is because at that point you're going back in to an age where relatively few readers were ever made anyway, only when going back to an age with relatively few readers made do you get scenarios where there really aren't any reader devices left. The data on the medium is what needs to be considered irreplacable, for anything within a century and a bit I think it is feasible to say that in the event you haven't maintained the reading system for yourself, someone somewhere will still have one you can hire. So it comes back to the question of what has the best integrity for the data ( 20GB to 100GB range, hence punched tape is a non-starter) and the medium on these sort of timescales."Software which can still interpret the old data formats will be yet another challenge..."This would usually be a major point, but given the data already includes the installer files necessary to set all this software up on a typical Linux distro, one is looking at a scenario where to recover in many decades all one needs is the medium itself, access to a PC which can have a circa 2023 Linux OS installed upon it, and the reading device. I think that the existence somewhere of the right reader device and of PCs from the 2010 to 2030 era, or simply emulation and/or virtual machines which could act like a circa 2023 PC as a layer running atop futuristic hardware, is something one can largely rely upon. Unlike individual pieces of data themselves, I think one can rely on the "crowd backup" scenario, that someone somewhere who can be contacted and offered cash to help, in regards to reading equipment and things like common Linux* distro iso files.*hence no future threat of a licence checking server somewhere meeting its demise and after that there being no way to activate software without paying absolute fortunes for the services of a very skilled reverse engineer if they could do it at all, software that depends on licence servers to activate or checks them periodically while running is a whole other can of worms which thankfully need not be considered here

author: nctnico
author_rank: Super Contributor
timestamp: August 17, 2024, 09:55:23 pm
content: I've come to the conclusion that the best way to actually keep data is to have it spinning on a couple of hard drives in parallel spread over several locations. If one drive goes bad or gets upgraded, copy the data over. This beats hardware and filesystem obsolescence AND it gets you active monitoring of drive health.

author: Someone
author_rank: Super Contributor
timestamp: August 17, 2024, 11:51:54 pm
content: Quote from: Infraviolet on August 17, 2024, 09:03:44 pmThere is no absolute best. More realistically what is the budget you envision for the project?I've seen these sorts of project many times before, and they usually fall over when x parameter must be >>> existing solution but cost must be ==. It is hard to beat mass production volume when value comes into it. Where is the world's data being stored? Spinning disks.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 18, 2024, 12:02:18 am
content: Quote from: Infraviolet on August 17, 2024, 06:47:43 pmDo you have evidence for that? That's not what I've seen.Quote from: Infraviolet on August 17, 2024, 06:47:43 pmLG ones seem to work fine on Linux, at least the internal, SATA.That's something I've been meaning to test for a while - will probably do at some point.

author: David Hess
author_rank: Super Contributor
timestamp: August 18, 2024, 01:17:09 am
content: I think M-Disc is still your best option.

author: Gribo
author_rank: Frequent Contributor
timestamp: August 18, 2024, 02:31:06 am
content: Encrypt it, and store it as stenographic pictures on Facebook/Instagram/whatever..

author: Rerouter
author_rank: Super Contributor
timestamp: August 18, 2024, 02:38:47 am
content: Its not at all cheap for that kinda data amount, but its sounding like NOR flash with some redundancy, and the write pin switched to lock after the initial write,Most other physical media will suffer from standard standard shift over that kinda time frame so you need something that can be read really easily and is so simple that its less likely to be replaced. Its not exactly easy to find a new 5" floppy drive still in working order.USB so far is the more likely standard to survive some backwards compatibility, but memory card interfaces are another promising one, after all people are still using film cameras,

author: tooki
author_rank: Super Contributor
timestamp: August 18, 2024, 02:55:53 pm
content: What's wrong with just using multiple hard drives and periodically copying to new drives? The important thing is to have multiple copies that are totally independent of each other. Ideally, you would use different drive models, replaced at different times, to mitigate the risk of a bad production batch that fail prematurely at the same time.This method is also how you ensure ongoing compatibility with modern hardware. Realistically, there's just no way to expect to save to a medium today and guarantee you'll be able to read it in 70 years. The media needs to survive, as does the drive, as does the system the drive connects to.Depending on your goals, you might want to copy not the files themselves, but rather entire disk images containing whole file systems. This can be useful if you need to e.g. restore an entire setup back to a physical drive (or to use the disk image for a virtual machine).This reminds me, I have some old stuff archived on CD-R that I should probably just copy as disc images...

author: nctnico
author_rank: Super Contributor
timestamp: August 18, 2024, 03:00:48 pm
content: Quote from: SiliconWizard on August 18, 2024, 12:02:18 amDoes blu-ray still exist?

author: BrianHG
author_rank: Super Contributor
timestamp: August 18, 2024, 03:11:40 pm
content: Quote from: nctnico on August 18, 2024, 03:00:48 pmBest way to enjoy 4K UHD movies. Bursts up to 200mbit for playback, over 200gb storage per disc. Each 200gb disc costs less than 5$ to press.

author: squadchannel
author_rank: Frequent Contributor
timestamp: August 18, 2024, 03:25:22 pm
content: Flash memory are very susceptible to quality. I have seen some that volatilize data in a few years.Unfortunately, BD is out of date.What about LTO tapes? You can get them cheaply if they are a few generations old.Panasonic is discontinuing BD media.Sony seems to be phasing out production.

author: Phil1977
author_rank: Frequent Contributor
timestamp: August 18, 2024, 03:29:56 pm
content: A fail safe passive storage medium for 100 years definitely is difficult.But if you actively care for your data then the combination of HDD + cloud should be the safest. Alternatively to the public cloud you can of course use any remote storage that´s available to you.If you check accessibility of the data e.g. based on a yearly schedule, then the risk of failure at both locations should really be small. Much smaller than the risk of any non-redundant technology can be.

author: PlainName
author_rank: Super Contributor
timestamp: August 18, 2024, 04:08:02 pm
content: Quote from: nctnico on August 17, 2024, 09:55:23 pmThis. If it ain't live it's dead and forgotten.Suffered several instances of storage media failing, readers becoming broken and unavailable, drivers no longer existing, etc. So anything needing to be kept is on a live system and thus gets into the normal backup regime. Reading isn't a problem, filesystems not a problem, obsolete hardware not a problem, having to think about how it's surviving not a problem.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 18, 2024, 09:15:43 pm
content: Quote from: nctnico on August 18, 2024, 03:00:48 pmUh, of course.

author: Alex Eisenhut
author_rank: Super Contributor
timestamp: August 18, 2024, 11:54:08 pm
content: Quote from: nctnico on August 18, 2024, 03:00:48 pmLong live HD-DVD!

author: Infraviolet
author_rank: Super Contributor
timestamp: August 19, 2024, 12:16:44 am
content: Has anyone experience with M-Disc burning then? With a USB connectable burner of some sort? M-disc's own site hasn't been updated in years, and the only M-disc DVD media still for sale are old one's being sold at massively marked up prices (relative to what they were when in production), so it looks like M-disc blu-ray is the only M-disc left. I had seen rumours that M-disc blu-ray is less resilient than M-disc DVD, because it has to somehow make the "features" involved smaller and more vulnerable so as to cram more data in to the same disc area, but these were indeed rumours. If anyone has hard data, on the still-available blu-ray M-disc types, or any tips about Linux compatible external USB burners, I'd be very interested.As for the redundancy NOR flash suggestion, are there long retention time NOR flash storage chips available at the gigabyte per chip sort of level?As for the encrypt and steganograph in to shared images, the idea is quite good for smaller data volumes. But I'm not sure if many sites would let you upload enough images to hold 10s of GB within them, and uploaded images sometimes get subjected to compression which would destroy any extra data inserted in them. Plus, a lot of those sites delete everything uploaded by any account that has been inactive for more than a particular number of years.Thanks

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 19, 2024, 12:59:20 am
content: No experience yet with M-DISC as I said, although that was on my to-do list.It would appear that the rumours you talked about originated on Reddit. Just found this video which deals with it:while inconclusive, it seems none of the arguments that were given hold. Still doesn't mean that current discs sold as M-DISC are "genuine", but apparently Verbatim denied that the recent ones would be fake/rebranded regular discs.Would be interesting to see a recent test of recent Bluray M-DISCs, unfortunately I didn't find much. Mostly old tests dating back several years.I also mainly use Linux now and wanted to give this a shot. I was leaning towards an internal writer (LG WH16NS40 typically) which was tested compatible with Linux. But I also would now prefer an external writer if possible. These of course tend to be much more expensive and possibly less mechanically reliable.I wouldn't trust Flash memory for long-term archival, especially if it just sits unpowered for long periods of time.

author: rsjsouza
author_rank: Super Contributor
timestamp: August 19, 2024, 01:38:50 am
content: TBH, for just 100GB of absolutely critical data I would buy several cheap(er) HDDs with at least CMR and create such redundancy that I would not worry about losing one or another disk. If needed, I would have around a few SATA to USB enclosures to have this data accessible in case the SATA itself goes away from newer machines. USB seems to be more resilient to interface changes.As others have said, it is unrealistic to believe this will be still operational decades from now if absolutely zero maintenance is provided to this dataset.

author: BrianHG
author_rank: Super Contributor
timestamp: August 19, 2024, 02:02:45 am
content: Quote from: rsjsouza on August 19, 2024, 01:38:50 amMake sure you use several different brands.You never know, stick with one brand, and for some odd reason the type of lubricant they use on the bearings seizes up after 25 years, then all the drives would be compromised.I would also add SLC flash drives, if you cannot find them, then at least try to find MLC and refresh them every decade or so. Stay away from TLC and up.Maybe store you data on a few flash EEPROMS which still use SLC cells.In 100 years, you will still find those who either kept or made their own eeprom programmers for historical restoration of antique devices like arcade video game consoles and ancient PCs.

author: Halcyon
author_rank: Unknown
timestamp: August 19, 2024, 02:28:36 am
content: Quote from: Infraviolet on August 17, 2024, 06:47:43 pmIn all seriousness, it's not a terrible idea for small amounts of data, but yes, Gigabytes are probably out of the question. This is an interesting project:https://ollydbg.de/Paperbak/Depending on how compressible your data is, you can store up to 3 MB on a standard A4 sheet of paper.

author: BrianHG
author_rank: Super Contributor
timestamp: August 19, 2024, 08:06:43 am
content: Quote from: BrianHG on August 19, 2024, 02:02:45 amI just found out there is a tool to modify the firmware of some TLC/QLC drives which forces them to run their flash ram exclusively in SLC mode. Though, the drive becomes 1/3 or 1/4 the size. But is you want more secure archiving, it is a way around the fact you cannot buy SLC drives anymore.See here:https://www.eevblog.com/forum/chat/does-anyone-make-good-ssds-any-more/msg5610277/#msg5610277

author: PlainName
author_rank: Super Contributor
timestamp: August 19, 2024, 08:09:47 am
content: QuoteNeat!Tried it, and... long story short, I printed to PDF then saved as PNG then converted to BMP. Apps says file format isn't supported, so perhaps modern BMP is not the BMP Windows used to love. In which case even this scheme wouldn't help to preserve data much longer than the OS it runs on.

author: Halcyon
author_rank: Unknown
timestamp: August 19, 2024, 08:23:08 am
content: Quote from: PlainName on August 19, 2024, 08:09:47 amPossibly, but at least image formats can be converted.

author: nctnico
author_rank: Super Contributor
timestamp: August 19, 2024, 08:45:53 am
content: Quote from: SiliconWizard on August 18, 2024, 09:15:43 pmInteresting. I thought these had died out years ago. Personally I never had a blu-ray disk in my hands. Either way, the removable optical media all suffer from the same problem: limited life span of the drives. Even in the era of CD-ROMs this was already a problem. A CD-ROM player would only work for a couple of years and then it needed replacing. I tried many different brands but they all had problems after several years.

author: BrianHG
author_rank: Super Contributor
timestamp: August 19, 2024, 08:49:04 am
content: With a 2400dpi film writer, you can print on rolls of film a gigantic QR code holding approximately 0.5 megabits of data for every 8.5x11inch page. Note that on the old AGFA imagesetter film writer my friend had, it could print 22 inches wide by a continuous roll for a large number of feet. (Minor improvements will need to be made to the QR standard to operate with such huge image sizes.)This means a QR code 22 inches wide, (minus safety margins, and upgrading to 3000dpi later models of the 2002 era), every foot printed will contain 2Gigabits of data, or 256 megabytes of data. This means 1 gigabyte for every 4 feet printed. Since the film is loaded in the imagesetters in a huge roll, something like an old scroll, you would need a film 22 inches wide, 400 feet long for 100 gigabytes of permanent storage.(LOL, a literal 100 gigabyte Torah...)You just need to have a good scanner to scan in the 22 inch wide film roll.(Example old AGFA imagesetter film printers, some working up to 3000dpi:https://www.exapro.com/sp/agfa-phoenix2250-5699/, note that since it is film, you do need the developer machine and the developing chemicals. But yes, the output is B&W film written by a laser, just like the film in your 35mm camera, just monochrome and 22 inches wide.)Example windows software app encoded into a QR Barcode, IE, you can make QR code any size you like, even holding computer software.

author: RoGeorge
author_rank: Super Contributor
timestamp: August 19, 2024, 09:13:34 am
content: There was a Romanian from the Institute of Atomic Physics at Magurele during the 90's or so, and he invented a many, many layers CD-like technology. The disk was the diameter of a CD, and about 1 cm thick (0.3inch or so), he did some demos with a prototype reader unit, and did some presentations to attract investors. Later he went to USA and sold the invention to some big corporation, Sony or Philips or such, don't recall the details very well.That technology was never mass produced, though since then, once each couple of years the tech news announce a petabyte optical disk was invented, like for example this one:https://www.livescience.com/technology/electronics/new-petabit-scale-optical-disc-can-store-as-much-information-as-15000-dvds

author: BrianHG
author_rank: Super Contributor
timestamp: August 19, 2024, 09:23:20 am
content: If you wan fire-proof and water-proof ~100gb storage on a ~4 inch square or glass, look at this technology which uses lasers to etch data into the middle of a square of ordinary glass which is designed to retain the data for 'thousands of years' unlike the cheap plastic, magnetic or electronic media we are used to:https://variety.com/2019/film/news/project-silica-superman-warner-bros-microsoft-1203390459/This tech should fit the OP's needs quite nicely.

author: jonovid
author_rank: Super Contributor
timestamp: August 19, 2024, 09:59:17 am
content: A history lesson in history its self-a more realistic way is to use data disks made of brass or ceramics to a new proprietary standard with an emphasis on longevity & durability not compactness or ease of use.so plastics would be avoided or anything that would decay or undergo decomposition over time.so for example this new data standard would use data disks made from pottery ceramics. that in theory can last thousands of years.putting data into a ceramic coating on a ceramic or pottery disc.the size circumference & thickness of any ceramic data disk would be a compromise between saving space & maintaining durability when handled with care.so saving total data integrity. one aspect of this is the disc playback device. that must be reconstructed from engineering plans also in ceramic tablet not paper.as this is disc playback device would not be expected to survive the passage of time.regrettably most of recent history going back the last 30 years could disappear with a catastrophic event. destroying the worlds data centers.even simple economics can destroy digital history. so for example saving space at a data center for new data. replacing old hard disk drives.at the end of the day when all is said and done IMO. text & images fired on ceramic tablets that can be visible to the naked eye.& also stored in an underground cave will be expected to survive the passage of time.I love electronics but do not expect it as we know it today, to survive the end of today's modern civilization.so for example in the last 30 years I have seen the loss of knowledge & skills that was once commonplace . the decay of common knowledge if you like.we as a civilization are building a digital pyramid but we no longer comprehend how the foundation was laid. we built this code on top of old code.with ever increasing in coding complexity like a spiral curve towards the centre. life gets too complicated going beyond human comprehension.at this point any catastrophic event can snuff out this pyramid of digital knowledge we call the internet. the Alexandria of our time.then no one understands how to fix it or decrypt it. large parts of the internet may become bricked when a software engineering greybeard diesparticularly if the internet AI can nolonger repair its self.and so preserving digital knowledge beyond one's life time is a challenge.

author: MT
author_rank: Super Contributor
timestamp: August 19, 2024, 11:32:26 am
content: Quote from: Halcyon on August 19, 2024, 02:28:36 amCuneiform clay tablets is the forerunner, grand dad of all "pressed" media oldest so far is from Gudea of Lagash, Sumer 2144-2124 B.C. Central Iraq.So a blue ray encoded clay disk which then burned into a ceramic and eventually glased for additional surface protection.Statue of Gudea I, dedicated to the god Ningishzida; 2120 BC with datat printed on his lap, base material Diorite.

author: PlainName
author_rank: Super Contributor
timestamp: August 19, 2024, 12:45:53 pm
content: QuoteBut still has the problem of acquiring a working scanner and appropriate drivers. The human eye just sees stuff, but the brain has to interpret what the eye sees, and if it's not in a language that is current then it may well be meaningless and effectively unreadable. The only reason we understand hieroglyphs and the like from ancient Egypt is because of clever people spending a long time decoding the stuff. For mere mortals it is write-only media, even if it lasts forever.

author: RoGeorge
author_rank: Super Contributor
timestamp: August 19, 2024, 12:50:40 pm
content: Writing 20-100 GB in cuneiform tablets, though, might take longer than 100 years.

author: MT
author_rank: Super Contributor
timestamp: August 19, 2024, 02:15:43 pm
content: Quote from: RoGeorge on August 19, 2024, 12:50:40 pmNot if its tiny, like the Chinese symbols can have 10 different meanings.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 19, 2024, 08:17:23 pm
content: Quote from: nctnico on August 19, 2024, 08:45:53 amNever had this problem. I still have a Plextor DVD drive/writer which works fine.But I admit I never actually bought a Bluray drive. I had one indirectly in my PS4, that's all.Sure these days with cheap external HDDs/SSDs and cloud services (and fast Internet access), the market for writable optical media has shrunk considerably. Still, I was also looking for a long-term archival media and optical still looks attractive, at least with M-DISC which looked promising. Haven't tested that yet though, and the company behind that disappeared (which triggered caution). Not sure what the deal is exactly now. I think Verbatim bought the technology, but it looks like nobody can really tell if Verbatim M-DISC is really as good as the original stuff.I think there's still a case (even if maybe limited) for long-term archival that would be purely passive (not requiring any maintenance / power / data replication / etc).

author: nctnico
author_rank: Super Contributor
timestamp: August 19, 2024, 10:28:44 pm
content: The primary problems with off-line / passive storage are 1) vendor lock-in and 2) hardware not being upgraded to the latest technology & OS support while staying backward compatible with existing media. Over the years I have had a whole bunch of off-line systems through my hands and none really stood the test of time. Hardware and media became obsolete for every system.

author: BrianHG
author_rank: Super Contributor
timestamp: August 20, 2024, 01:07:58 am
content: This is all BS.Just buy a pile of eproms and burn them with redundant checksum.I have eproms from the early 80s and they are all fine.Just make sure you over-burn each cell to get that extra endurance.In 100 years, even if you cannot find an eprom reader as historical hardware enthusiasts will still have a few for rejuvenating vintage technology, it is easy enough to build your own eprom reader assuming we still use electricity to drive IOs in electronics in one form or another.How difficult is individual address wires tied to a binary counter, and reading 8bit parallel data.

author: PlainName
author_rank: Super Contributor
timestamp: August 20, 2024, 09:12:50 am
content: Microchip do 8Mb EPROM, so for 100GB you'd need 1000 of them at £8 ($10) a throw in quantity. Need 25A to power them, but I guess you could arrange to do them in blocks to reduce that kind of requirement.

author: iMo
author_rank: Super Contributor
timestamp: August 20, 2024, 09:20:32 am
content: ..up to 1TB printed on an A4 double side

author: BrianHG
author_rank: Super Contributor
timestamp: August 20, 2024, 09:41:54 am
content: Quote from: PlainName on August 20, 2024, 09:12:50 amAt 3.3v, that's only ~100 watts. I do not think it will be a problem.Though, doesn't each one have a CE power-down state secondary to the output enable? That should cut down the current requirements at the expense of waiting the extra 100 or so nanoseconds as you enable a row/section...

author: Haenk
author_rank: Super Contributor
timestamp: August 20, 2024, 10:19:15 am
content: I'd say: Define your budget. If you are running on a "100 Bucks" budget, even thinking about a hardware-solution it is pointless.For low budget, I'd go with DVD-RAM or DVD-R - create multiple copies of your archive on different branded media, re-check every 5 years and create more sets. Store safely at different locations (cold, non-condensing, dark - maybe in a sealed metal-coated bag with drying bags).That should be cheap, and would require only a bit of effort once every couple of years.As I have plenty of 25 year old burnt CD-R, I would expect the lifespan of your backup in case of not being refreshed about the same timeframe; remember that there will be multiple copies availabe, which will reduce chance of a total loss even more.Of course you could go with higher density discs like M-Disc, but that might be a problem, if the reading devices are no longer available (go find one in your town); you likely won't have a problem to find a DVD reader.

author: nctnico
author_rank: Super Contributor
timestamp: August 20, 2024, 11:31:42 am
content: Still, if you take a step back and first define what kind of data needs to be stored. The OP never specified that. History has proven that texts and drawings on high quality paper can last for hundreds of years. And realistically, what kind of information is actually worth keeping? Public information like movies, books, records, etc are stored elsewhere already. So what is left are personal pictures, movies and texts. Pictures and text can be printed on high quality paper, digital movies can be converted into a film format used professionally. Both will last forever.For example, I have a book with illustrated stories made by my great-great-great grandfather nearly 100 years ago. The colors of the illustrations are as vivid like it was drawn yesterday.

author: PlainName
author_rank: Super Contributor
timestamp: August 20, 2024, 01:04:48 pm
content: QuoteYes, they can be. But, equally, they can last half an hour in the wrong circumstance. Also, we only know of the ones that made it, never the ones that didn't (observer bias).QuoteAnd likely to disappear on a whim. There are LOADS of films and books you can no longer access.QuoteIndeed, that would help enormously, even if only to open another rabbit hole

author: nctnico
author_rank: Super Contributor
timestamp: August 20, 2024, 01:10:17 pm
content: Quote from: PlainName on August 20, 2024, 01:04:48 pmTrue. But if you put a USB stick, a optical disk, an EPROM and a piece of paper in a safe, I'm pretty sure the thing that remains readable after 100 years is the piece of paper. The rest needs effort and knowledge that might no longer exist IF (big if) the medium hasn't degraded.

author: David Hess
author_rank: Super Contributor
timestamp: August 20, 2024, 03:06:50 pm
content: Quote from: nctnico on August 20, 2024, 01:10:17 pmThe EPROM, optical disk if not organic dye based, and paper will still be readable. The USB stick, or any current Flash based media, will be toast. Based on my tests a couple years ago, I would consider it lucky if a USB stick lasted more than a year.

author: PlainName
author_rank: Super Contributor
timestamp: August 20, 2024, 04:21:34 pm
content: Why are USB sticks so dodgy compared to, say, SDcards or SSDs?

author: ebastler
author_rank: Super Contributor
timestamp: August 20, 2024, 04:30:19 pm
content: Quote from: PlainName on August 20, 2024, 04:21:34 pmAre they? Not in my experience, as long as you buy brand name sticks.

author: nctnico
author_rank: Super Contributor
timestamp: August 20, 2024, 04:43:48 pm
content: Quote from: ebastler on August 20, 2024, 04:30:19 pmAgreed. Never had trouble with name brand USB sticks. Typically I buy the ones from Kingston from reputable sources.

author: PlainName
author_rank: Super Contributor
timestamp: August 20, 2024, 04:51:03 pm
content: Quote from: ebastler on August 20, 2024, 04:30:19 pmThey don't seem to have a very good reputation. Possibly not helped by being rather slow too.

author: tooki
author_rank: Super Contributor
timestamp: August 20, 2024, 05:43:54 pm
content: Quote from: PlainName on August 20, 2024, 04:51:03 pmThat's because there are tons of garbage thumb drives out there, and many people bought those (or got them as promotional gifts, etc).Thumb drives from top-quality vendors (the same ones one should be choosing for SD cards, too) just work. I've always bought SanDisk thumb drives, and SanDisk and Lexar memory cards (CF, MS, and SD, in my case) and have never had one fail.More expensive ones are faster. But it does seem that thumb drive Flash controllers were not historically optimized for speed like SD card controllers. That changed with USB 3, and of course a USB SSD is just a high-performance thumb drive.

author: PlainName
author_rank: Super Contributor
timestamp: August 20, 2024, 06:47:04 pm
content: A lot of USB3 sticks are not really any faster on writes. Read, perhaps, but writes still boringly slow.

author: David Hess
author_rank: Super Contributor
timestamp: August 20, 2024, 08:17:00 pm
content: Quote from: PlainName on August 20, 2024, 04:21:34 pmPrompted by mysterious incidents of data loss on USB Flash sticks, a couple years ago I ran tests are a bunch of random and almost new USB Flash sticks which I had collected, some generic and some name brand. All of them lost data within not much longer than a year. Some of the no-name ones lost data within a month. Whether the USB stick was powered or not did not seem to make any difference. These tests convinced me to use SSDs in USB enclosures for routine external storage instead of USB sticks, although this is not always possible.Thinking about it, I concluded that because USB Flash sticks cannot rely on idle-time-scrubbing or scrub-on-read, because they cannot have power loss protection, they do not retain data well even when powered. SSDs however may have power loss protection, so can support idle-time-scrubbing.I have had better results with SD cards, at least when they are in continuous use.There are some companies which make what should be reliable USB sticks and cards, like Swissbit, but they are also really expensive.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 20, 2024, 09:10:03 pm
content: Quote from: Halcyon on August 19, 2024, 02:28:36 amExcept "modern" paper is utter crap and won't survive past a few decades, contrary to some very old paper that is still almost intact centuries later.And of course, the minimum for the printing itself would be to use a laser printer, but even toner is not going to last longer than a few decades (depending on storage conditions). Maybe that's enough, but that can't be compared with some ancient paper sheets and inks that were made with completely different (and more durable) materials.Some argue that very long term archival doesn't make sense anyway because people are likely not to have the equipment to read the data, a few centuries from now (or even just a few decades). But as long as the medium/format is documented somewhere, they'll always be able to rebuild equipment to read it if needed (unless humanity has gone through so bad de-evolution that they'll be unable to figure it out, in which case, it doesn't look good anyway).

author: PlainName
author_rank: Super Contributor
timestamp: August 20, 2024, 10:18:47 pm
content: Quote from: David Hess on August 20, 2024, 08:17:00 pmThanks.Now I think about it I do have some USB sticks doing serious duty - they are the boot drives for some servers which run up Xpenology and ESXi. Typically they are read only (copy Linux into RAM to run) and always on. But there have been power cuts and other power-down situations, and so far I've not had one fail to boot. Might just be lucky

author: nctnico
author_rank: Super Contributor
timestamp: August 20, 2024, 11:42:36 pm
content: Quote from: SiliconWizard on August 20, 2024, 09:10:03 pmThat is why acid-free paper specifically for archiving and preservation purposes exists:https://www.preservationequipment.com/Catalogue/Conservation-Materials/Paper-Board/What-is-acid-free-paper(random Google find).QuoteIf needed... but needing to build a contraption becomes expensive quickly which makes it not worthwhile to recover the information. And for documenting the storage method, you'll be back to using paper.

author: 3roomlab
author_rank: Frequent Contributor
timestamp: August 20, 2024, 11:56:26 pm
content: interesting article about media lifespanhttps://www.canada.ca/en/conservation-institute/services/conservation-preservation-publications/canadian-conservation-institute-notes/longevity-recordable-cds-dvds.html

author: jonovid
author_rank: Super Contributor
timestamp: August 22, 2024, 01:25:11 am
content: perpetual generational backups every 10yrs is possible to keep it forever.this works for linear media as in order to play back video sound images or text.however porting raw unedited code or software between different hardware & media generations can be problematic.in preserving the essence of the original digital media applications.so for example porting a video game that is 20 yro software so it will run on new or next generation of hardware.generational business practices can be a threat. so for example planned obsolescence in all types of digital media.

author: tooki
author_rank: Super Contributor
timestamp: September 07, 2024, 10:57:18 am
content: Quote from: SiliconWizard on August 20, 2024, 09:10:03 pmI think you are somewhat misinformed.Certainly there is no evidence that “modern” (in the sense of what we can buy now) office papers only last a few decades, and we have papers from every era survive perfectly fine. In fact, papers from the early 20th century ageworsethan more recent papers, since we learned from the problems with early-20th-century papers.In the 1980s, all halfway-decent papers were switched to acid-free formulas/processes, eliminating the source of most deterioration: acidity. (And in fact, there are bulk processes to deacidify paper that isn’t acid-free. Archivists use those methods to stabilize entire archives of documents printed on pre-1980s office paper.)The papers that survive really poorly are the extremely cheap papers used for newspapers and inexpensive paperback novels; those start to yellow after just a few years in suboptimal conditions. But nobody wouldeverselect those as an archival medium! You’d use a quality archival paper.And it’s not as thoughallold/ancient papers and inks survive well. Archivists and historians have to deal with deteriorating documents all the time. Many of the inks used in antiquity are really awful, and either fade or damage the paper, or both. The common iron gall ink that was the dominant ink used for over 1400 years in the West is acidic, and sometimes eats through the paper. (Modern iron gall ink is formulated differently so that the acid evaporates quickly, reducing the problems. It’s a niche product now, though.)As for what printing method to use: toner is good if the paper is kept cool and above allnot in contact with soft PVC, whose plasticizers leach out, causing the toner to adhere to the PVC and release from the paper. In particular, monochrome laser is excellent. But honestly, if I were choosing a printing method, I would probably use inkjet. Modern pigment-based inks are very stable, and are waterproof. Since they absorb into the surface of the paper a bit, they can’t easily delaminate. (But one must choose an inkjet with ISO archival compliance, not just any old whatever from Best Buy.) I could also imagine that someone could make (if there isn’t already!) inkjets with specialty inks that adhere extra well and perhaps are also solvent-resistant.Archival inks are a known technology. (Germany, for example, has specific standards for archival inks in pens, and requires many types of important documents to be signed with pens that have archival inks. There are similar standards for printer inks and toners.)There are gazillions of toner-based photocopies from the 60s and 70s that survive just fine. Even those stored in mediocre conditions are usually okay. (Other than the aforementioned contact with PVC.) Heat and pressure can cause toner to slowly release to adjacent sheets, especially if the fusing process wasn’t ideal when the print was made. Stored dry, cool, and dark, toner survives well.So if you’re talking about archiving historical documents and items originally intended for everyday use, then conservation of materials with poor archival characteristics is a big problem. But the context of this thread isdeliberatearchiving, with careful selection of materials for that purpose. There is no reason to doubt the longevity of these.

author: rsjsouza
author_rank: Super Contributor
timestamp: September 10, 2024, 12:05:52 am
content: Indeed. I have 125+ year-old books that will beg to differ from the misconception that printed paper is crap. Sure, the pages do not exhibit anymore their flexibility and can get quite discolored, but its readability is still quite optimal. My earliest electronics-related material is dated back to 1952 and is in excellent shape until this day - all this is even more impressive as it was stored in quite sub-optimal conditions during its lifetime.

author: Halcyon
author_rank: Unknown
timestamp: September 10, 2024, 02:08:09 am
content: Quote from: ebastler on August 20, 2024, 04:30:19 pmI wouldn't go as far as saying "sododgy", but based on my experience, I've had more failures of USB flash drives than any other media type (out of the many hundreds of flash drives I've used).I generally buy Sandisk, but even then I've had a few of those die (including one of their "Extreme Pro" series). I've also had a few Verbatim drives go bad.That being said, I also have an old LaCie USB 2.0 drive which has been fine for the last 20 years.If you buy the cheap Chinese rebranded/no brand crap, then I wouldn't trust those as far as I can throw them.Either way, I typically treat all USB flash media as a temporary storage only and certainly don't use them to store anything I can't afford to lose.

author: Phil1977
author_rank: Frequent Contributor
timestamp: September 10, 2024, 07:03:25 am
content: Quote from: Halcyon on September 10, 2024, 02:08:09 amIt´s the usual "application rank":The highest quality memory chips go into embedded systems, server SSDs and fast SSDs. Then come high quality storage cards like CF-Express cards and SATA-SSDs. Then come SD-Cards - USB Sticks are last before the trash bin.In one of the OEMs specification this was shown in kind of a "quality pyramid".

author: Halcyon
author_rank: Unknown
timestamp: September 10, 2024, 07:10:38 am
content: Quote from: Phil1977 on September 10, 2024, 07:03:25 amYep, 100%. And it's that age old "your get what you pay for". Don't expect $500 performance/resilience/reliability out of a $50 device.

author: Phil1977
author_rank: Frequent Contributor
timestamp: September 10, 2024, 07:43:18 am
content: If I sell you this trashy USB-Stick for $500,- I´ll promise it to be very good

author: Ranayna
author_rank: Super Contributor
timestamp: September 10, 2024, 07:59:00 am
content: Quote from: David Hess on August 20, 2024, 03:06:50 pmFor anything requiring anything special to read - and maybe more important - understand! the archived data, you will run into trouble.Printed paper may already be iffy if languages change, though this will not happen in a hundred years.But even if you can still read EPROM, or read a disc with a microscope, can you still understand the encoding?

author: macboy
author_rank: Super Contributor
timestamp: September 10, 2024, 02:45:20 pm
content: I still have and use some TDK brand Archival DVD+R media. They claim a 100 year lifetime (special dye?) and have the same ultra-hard coating that Blu-ray disks have. Running one's fingernail on the bottom surface of the disc, the feeling is distinctly different than regular DVD or CD media. It feels hard like glass, like a phone screen, rather than plastic. You literally can't scratch these with your fingernail, and they survive a lot of mishandling without damage. These are single layer 4.7 GB discs but use dual-layer-like construction, sandwiching the recording layer in between two substrate layers, rather than having it on the top of a single substrate and coated with a thin "protective" coating like standard CD and DVD media. This physically protects the recording layer and slows the (inevitable) ingress of damaging oxygen. Because they are "dual" layer and DVD+R (not -R) they can't be read on some of the oldest DVD-ROMs which lack those capabilities. Every so often, I burn some irreplaceable data onto some of these discs and stash them away. TDK no longer makes these but Verbatim has a similar product. Between the construction, the 100-year dye, the superior DVD+R error correction and laser tracking (vs DVD-R), and the scratch resistant coating, these disks are in a different league from regular DVD-R blanks.I also keep a single special old DVD-ROM drive (actually a "combo" drive: burns CD but only reads DVD). I recognize it as special because I once, several years ago, was trying to copy data from some very old CD-R discs, but had a heck of a time reading them. Neither drive in my main computer would read the discs, nor would the drive in my secondary computer nor laptops (remember when every laptop had an optical drive?). Some just spun and never mounted, some would mount but read slowly then error out. As a last resort, I tried all the drives I had stashed away, and amazingly this drive was able to read every single bad CD-R without any issue, and at full speed! I promptly labelled this drive as special and have kept it ever since.I wish I had a better off-site backup strategy, but I'm still learning and growing too.