The Story of How Our Data Can Be Stored Forever: From Microform to Macromolecules
By: Eurecom Communication
Tags: Eurecom, Innovation, Research, Dna Storage, Database

We are producing more data in the last few years than we ever did as a human kind. We now store all of our thoughts and actions as digital files. However, except when we run out of storage space on our phones, we do not usually think much about problems related to storing our data over several decades. IBM manufactured the first hard drive in 1956, where you could store one mp3 song in a disk at the size of a washing machine. We have come a long way and data storage technology has evolved tremendously over the last five decades. Despite this, no storage technology available today is able to keep up with the rate of digital data production, or provide long-term storage of data at a low cost. What if there was a storage technology with the capacity to store a huge amount of data in a tiny space and essentially last forever? Well, actually, there is such a medium inside every cell in your body; it’s DNA! DNA is the oldest storage medium, that encodes all the necessary information for life to continue and evolve, in an incredibly tiny molecular package. Our data are our story, and in order to safely preserve it for future generations, alternative storage media like DNA might be just what we need.

EURECOM’s professor Dr. Raja Appuswamy, expert in Databases and Storage Systems will guide us through, on how analog media (microform, film, archival paper) and biological media (synthetic DNA) can be used for storing digital data and what challenges we have to face, until this technology can be used in large scale.

Q. What is the actual problem for data storage?

RA. Today, magnetic tape is the gold standard when it comes to long-term data storage across all disciplines, from Hollywood movie archives to particle accelerator facilities. It is highly denser and less expensive compared to alternatives like Hard Disks or NAND flash. However, magnetic tape medium suffers from some fundamental limitations, making it unsuitable for long-term data storage. First, it has a limited lifetime of only ten to twenty years. Thus, using tape for archival storage implies the need for data migration to deal with device failures. Second, the storage density of tape, or the amount of data you can store per square inch, is improving at a 30% rate annually. Thus, the tape industry releases a new generation of tape every 3 to 5 years. However, tape libraries and readers retain backwards compatibility only up to two generations. Therefore, using tape for long-term storage implies constant data migration in order to deal with technology upgrades. These migrations cost millions of dollars and put an enormous strain on archives and institutions that have to preserve data over several decades. For instance, Hollywood movie archives have already admitted that we are living in a dead period during which the productions of several independent artists will not be saved for the future due to skyrocketing data migration costs.

Q. Digital devices for storage are limited in time duration apparently. Is there no other device currently with a lifespan of a few hundred years?

Digital data printed as barcodes to film

RA. Of course, there are media that have a long lifespan. For instance, analog media like microfilm, microfiche, and cinema film are very durable and are expected to last up to 500 years. However, while such media have been used for preserving movies and journals across several generations, it was not clear if they could be used for archiving born-digital data. What we managed to do in a collaborative initiative between EURECOM and EUPALIA, a startup specialising in digital preservation, is to show how digital data can be efficiently stored in such visual, analog media by mapping bits into barcodes that can be printed on microform, film, or even archival paper. The system Micr’Olonys, which is a commercial solution from EUPALIA, is capable of storing 1.3GB of uncompressed data in a single 66 meters long, and 16mm wide film. Thus, analog media might be an ideal solution for enterprises with small or medium-sized archives, where a reasonable stock of a few hundred film rolls could be used to archive data for hundreds of years. Now, if a company’s needs exceed several terabytes, then analog media are not the ideal solution since they are not dense enough. For massive data storage, we believe that synthetic DNA might be a good fit.

Q. It is very exciting to work on several solutions accommodating different data storage scales. Now, why DNA is ideal for massive storage and what problems does it solve that other media fail?

Digital data stored as DNA in DNAShell and as barcodes on paper

RA. Using DNA for digital data storage is an old idea that has been around since the 1960s. Richard Feynman even pointed out in the 60s that DNA could be used not only for storage but also as computational building block.

The main advantages of using DNA as a storage medium are the following:

Density: in 1 gram of DNA we could store 455 Exabytes of data. In contrast, a 3.5” hard disk drive can store 10 Terabytes and weighs 600 grams today. Relevance: The density of DNA is fixed by nature, and we will always have the ability and the need to read DNA–everything from archeology to precision medicine depend on it. Thus, DNA is an immortal storage medium, which does not have the media obsolescence problem and hence, can never become outdated, unlike other storage media. Durability: DNA can last several millennia as demonstrated by experiments that have the read DNA of ancient, extinct animal species from fossils that are dated back thousands of years. If we can bring back the woolly mammoth to life from its DNA, we can store data in DNA for millennia.

Given these benefits, we believe that DNA has the potential to serve an excellent archival media for storing digital data over long time periods.

Q. What is the procedure for DNA storage and what are the technical challenges EURECOM provides solutions for?

RA. Let me describe briefly the procedure of storing data in DNA molecules. DNA is a macromolecule that is made up of four types of smaller molecules called nucleotides: Adenine ( A ), Cytosine ( C ), Guanine ( G ), Thymine ( T ). So, to store digital data on DNA, we need to convert it from a binary code (sequence of bits) into a quaternary code (sequence of nucleotides). At EURECOM, we have a long history of research in developing optimal codes for classic telecommunication channels. Building on this expertise, we use concepts from coding theory to develop error-correcting codes, that can encode and decode digital data by treating DNA as a storage channel. Once the binary pairs are encoded to an A, C, G, T sequence, the DNA is physically generated using a process called “Synthesis”. We are collaborating with synthetic biology companies for this purpose. We then store the synthesised DNA in DNAShell (see picture above), a container developed by Imagene technologies that allows us to store DNA at room temperature for a long time period.

Sketch of the process of DNA storage of digital data

The reverse procedure for reading the digital data back from DNA starts with a biological procedure called “Sequencing”. The synthetic DNA is sequenced using a procedure very similar to that of sequencing human DNA. The output of sequencing is a massive number (hundreds of millions) of “reads” — sequences of ACGT “strings”, from which we need to decode back the original digital data. Both synthesis and sequencing are not 100% accurate. Thus, the DNA sequences that we feed as input to the synthesis step, are not the same ones we get as the output from sequencing. Some DNA sequences are overrepresented in reads with multiple copies, some have errors with spurious nucleotide being inserted, deleted, or substituted, and some DNA sequences are not present at all. To overcome these errors and recover the data, we use several algorithmic, statistical, and hardware acceleration techniques to enable fast, accurate recovery of data. Interestingly, some of the techniques that we developed for retrieving data from DNA turned out to be very much applicable solutions to problems in other domains, such as the sequence alignment step in genomics-based DNA sequencing pipelines. Thus, we are building computational biology tools to assist researchers in these other domains as well.

Q. What is the status of DNA storage now and what can we wait in the near future?

RA. Right now, the major limitation facing DNA storage is the synthesis procedure, as it is quite slow and expensive, making DNA about 100,000 times more expensive than magnetic tape. We would need radically innovative technologies for making synthesis efficient, cheaper and faster. Several synthetic biology start-ups are investigating promising, novel, synthesis techniques in a race to bridge this gap and certainly the company that solves the problem of DNA synthesis, will be at the forefront of a new revolution in computing. Being conservative, I would say that we are 10 years away from large-scale adoption of this technology. However, in the meantime, DNA storage is likely to be used in niche domains for preserving artefacts of cultural or historic significance.

Q. And a more practical question for the end! How can we guarantee that in a 1000 years a future user that would come across our data, would be able to access them? Technology will essentially be different…

RA. We are also actively working on this issue in collaboration with EUPALIA! After developing encoders and decoders for microform and macromolecules, we realised that we also need to archive the decoders in addition to data, in order to be able to guarantee successful recovery. The central idea we are working on, is to encapsulate the decoders in an emulator in such a way that the emulator itself can be stored together with the data. A user 1000 years in the future would need to write a very simple algorithm to bootstrap the emulator that would automatically execute the decoder and recover back the data.

Interested in knowing more about how someone in the very distant future can use emulation to recover data? Let’s find out how this is possible in our next article!

Thanks Dr. Raja Appuswamy!

by Dora Matzakou for EURECOM

References