Last week, I wrote an article about how keynote speaker Martin Leach presented a convincing argument to Bio-IT World Conference 2012 attendees here in Boston as to why the biggest obstacle facing the health and life sciences industry in the age of "big-data" is not one of storage, but of computing.
Accessibility, analysis, and integration are the sole true bugaboos, says Leach, making storage issues but a petty distraction when it comes to genomics and others who work with intensive bioinformatics.
Robert Bjornson is director of IT at the Yale Center for Genome Analysis (YCGA). "We spend almost no time thinking about computing. We spend all of our time thinking about storage," told Bjornson to a room of a few dozen conference attendees.
In a presentation about IT infrastructure and hardware, Bjornson talked about the technological challenges YCGA and similar organizations face.
Even Leach does not dispute this fact of IT life. The Broad Institute of MIT and Harvard, where Leach is CIO, boasts the largest genomic datacenter in the world, with over 10 petabytes of data on spinning disks -- and every day to day-and-a-half, one of those disks fails.
"When you have 1,000 drives, expect failure," confirms Bjornson, by way of advice. What's more, backing up all of a genomics organization's data -- which can number in the petabytes -- just isn't practical.
Cost is also a factor (Moore's Law notwithstanding) for some customers, says Bjornson -- at least psychologically. Despite the price of storage falling, many enterprise and high-level organizational customers maintain a consumer market perspective. "I can't tell you how many times people have said, 'Why does this cost $1,000 a terabyte?' " says Bjornson, relating laughable characterizations of customers who protest that hard drives at Best Buy can go for about $65 per terabyte.
Big-data customers may be their own worst enemy in more ways than one. YCGA's customers use YCGA's storage and YCGA's cluster. Cautions Bjornson, however, "It's risky to let customers into the factory." They can crash the login node. They can overload the storage. They can "do any one of a number of things that people do when they get the chance," Bjornson says, and any of those things can interfere with their data management and data analysis.
To be fair, this is an example of a risk that falls under both the "storage" and the "accessibility" umbrellas -- and there are others.
For instance, Bjornson himself concedes that search is a huge problem in big data genomics, as he presents a slide that reads, " 'Find' does not work on 2PB on Storage." Genome sequencing, of course, is a data-intensive field -- yet the field of genomics lacks a truly effective data identification solution ("a Google search for data," as Leach called it on Tuesday), says Bjornson. "We don't have it. We really need it."
Both sides of the accessibility vs. storage discussion raise very valid points -- and have very real concerns. Alas, the hardware presentation series here at Bio-IT has been somewhat sparsely attended compared to other sessions. Conversely, so many Bio-IT attendees clamored to see the opening keynote speeches on Tuesday that dozens were relegated to an overflow room with a live video feed.
With so many of the attendees here having heard only Leach's advocacy for accessibility, arguments like Bjornson's about the importance of storage seem to have become lost in the din of the conference -- and therefore, ironically, much less accessible.
@white.space, I remember the old statement, "Nature abhors an empty horizontal surface." I think that we could also say that, "Nature abhors an un-filled storage bit," and we'd be just as correct. I've watched storage requirements grow exponentially during the last 30 years -- I can hardly wait for the day when the desktop petabyte is commonplace!
Joe, any potential outcomes for bioinformatics collaborative projects, for drug discovery. I heard that there are some proposals for TB, HIV and cancer drug discovery.
Taimoor, we had similar storage constrains for the Bio informatics projects and it's a collaborative project. What we had done is we formed a virtual group and create some common repository, where we kept all the datas. So those who are interested can access the data at any point of time, irrespective of location through net. This can help to avoid storing the same data at multiple locations.
Incidentally, this is why Netflix was pretty unaffected by the huge AWS cloud outage a year ago even though they're a major AWS customer -- because they were smart enough as an organization that handles enormous amounts of data to have so many redundancies that they could handle an outage. It was the little guys who relied completely but without backups on the cloud who were screwed for the following days.
Very nice breakdown, zerox, and rather one of the points I was going for. Both concerns ought be heard and addressed for real progress.
A recurring theme at the Conference was the unsuitability of the public cloud for a lot of bioinformatics work -- especially in the field of genomics -- because of the TREMENDOUS amounts of data. Far too much to be sending across public cloud data lines.
And, of course, there are issues with proprietary data, HIPAA, and other confidentiality (this one's for you, Sara!) bugaboos.
This reminds me of a discussion from a Dell-sponsored Webinar E2 hosted quite some time back in which the speakers discussed how many IT Departments are most concerned with just keeping the lights on -- more than anything else.
Perhaps that is another part of YCGA's struggle. Sure, better integration and accessibility would be nice, but it's a huge effort just to deal with what they have.
Absolutely! You could have posters and bumper stickers with that adage! I bet you have a Dropbox, and a Google Drive account (at the very least!), and several external hard drives filled with stuff.. :) And still wish there was more. The library of Congress has roughly 20Tb worth of stuff, and sometimes I wish I had as much space!
@Skr2011- And the opposite seems to be true, too. If we find we have some extra space, someone is going to come up with something to fill that space.
"Passwords are dead," a Google information security manager decreed at last month's TechCrunch Disrupt. Other pundits have come to the same conclusion. However, these reports are greatly ...
In this episode, you'll learn how to stretch the limits of your private cloud -- and how to recognize the limits that can't be exceeded.
IT has to deploy Server 2012 in a way that fits the architecture of its application delivery system.
Welcome to the backside of the virtualization wave. Discover the state of virtualization management and where analysts are predicting it is heading
Case Study: Boston Medical Center cuts data center power bills by 50% and saves up to $3.2 million with virtualized storage, server and desktop solutions
Case Study: Henrico County Public Schools realizes improved tests scores and saves $1 million using Dell systems with Intel processor technology
Niagara College enhances computing flexibility while consolidating IT by 40 percent with help from Dell Services
Automate storage tiering simply and efficiently with Dell EqualLogic unified, virtualized IP-based storage.
At the GigaOM Structure conference, a startup announced a cloud and virtualization storage optimizing approach that shows there's still a lot of thinking to be done on the way storage joins the virtual world.
We asked CIO Steve Rubinow what CIOs in other industries can learn from the financial services industry about datacenter efficiency, security, and green computing.
We ask CIO Steve Rubinow what pieces of financial services infrastructure need to perform better to get traders info faster.
We ask CIO Steve Rubinow: What do stock market traders need to know, how fast do they need it, and how can CIOs get it to them?
Intel supplies the best ingredients to drive greater datacenter efficiency and support new compute, storage, and networking needs.
Dell’s Gaurav Chand says that for the first time ever datacenter technology is truly enabling all kinds of organizations to transform their business and achieve new objectives.
We always hear about "Big" data, but a real issue in cloud storage is not just bigness but also persistence. A large data model is less complicated than a big application repository that somehow needs to be accessed. The Hadoop send-program-to-data model may be the answer.
EMC's Project Lightning has matured into a product set, and it's important, less because it has new features or capabilities in storage technology and management, than because it may package the state of the art in a way more businesses can deploy.
Oracle's release of a Hadoop appliance for Big Data may be a signal that we're shifting to database appliances.
We're at the beginning of a cloud-driven revolution in storage, but Oracle's quarter shows that enterprises are hunkering down on old concepts because they're afraid of the costs in the near term.
Untrained end users, clueless central IT staff, and expensive mobile devices are a worrisome combination for healthcare CIOs.
However you label it, structured and unstructured information are different and will likely always require different tools.
Need strategic guidance on everything from SSDs to 100 percent virtualized datacenters? Look no further.