These databases integrate data from many sources, and transform that data using a mixture of computer programs and manual annotation. Thus they become so complex as to require systematic empirical study, such as to measure the error rate within a database, and to verify whether the database is internally consistent. There are not enough CPU cycles for biomedical researchers. The NIH should support, in concert with NSF, localized super computing based on scalable clusters of commodity processors. The goal should be to provide scalable clusters, data servers, and high-speed network access in every biology department.
Scalable clusters require significant expertise for their configuration and operation.
Therefore, NIH should establish a clearinghouse for support expertise and resource scheduling software. Moreover, some important large projects require resources well beyond the capabilities of what is likely to be found in most biology departments. NIH should support large centralized resources, built on scaled-up versions of the same commodity processors, and with robust software and experienced staff, to enable such projects to be carried out in a timely fashion. An important issue for NIH will become the conversion of academic software to production-grade. The issues involved are not at all clear-cut, but have profound implications for bioinformatics software usability.