Abstract-We believe computational science as practiced today suffers from a growing credibility gap -it is impossible to replicate most of the computational results presented at conferences or published in papers today. We argue that this crisis can be addressed by the open availability of the code and data that generated the results, in other words practicing reproducible computational science. In this paper we present a new computational infrastructure called RunMyCode.org that is designed to support published articles by providing a dissemination platform for the code and data that generated the their results. Published articles are given a companion webpage on the RunMyCode.org website from which a visitor can both download the associated code and data, and execute the code in the cloud directly through the RunMyCode.org website. This permits results to be verified through the companion webpage or on a user's local system. RunMyCode.org also permits a user to upload their own data to the companion webpage to check the code by running it on novel datasets. Through the creation of "coder pages" for each contributor to RunMyCode.org, we seek to facilitate social network-like interaction. Descriptive information appears on each coder page, including demographic data and other companion pages to which they made contributions. In this paper we motivate the rationale and functionality of RunMyCode.org and outline a vision of its future. Terms-reproducible research, reproducible computational science, dissemination platform, collaborative networks, cloud computing, executable papers, code sharing, data sharing, open science Over the previous 20 years the practice of science has undergone a revolution, that will finish with computation as absolutely central to the scientific enterprise. Today's academic scientist is more likely to resemble a computer jockey working at all hours to launch experiments on computer servers. Long gone is the traditional image of the scientists as a solitary person working in a laboratory or scribbling in the back aisles of the mathematics library. This transition has not been smooth, and we believe it has already brought us to a state of crisis. The vast majority of scientific results generated by current computational science practice suffer a large and growing credibility gap: it is impossible to replicate and verify most of the computational results published or presented in conferences today. There have traditionally been two branches of the scientific method, the deductive and the inductive. The deductive branch encompasses fields such as mathematics and logic, and the inductive field comprises the empirical sciences. There has been much discussion, even at very high policy levels, over the last few years regarding the emergence of new branches of the scientific method arising from the computational revolution [1] . Both cpu-intensive simulation and the data deluge are provoking discussion of third and fourth branches of the scientific method, but we cannot elevate computational science to a new branch of the scientific method until we can generate reliable verifiable computational knowledge. The central motivation for the scientific method is the ubiquity of error -the phenomenon that mistakes and selfdelusion can creep in absolutely anywhere in the process of generating scientific findings, and that the work of the scientist is primarily about recognizing and rooting out error. Both the deductive and inductive sciences are error prone and have consequently developed standards for the dissemination of results. In the deductive sciences the formal notion of the mathematical proof is a mature response to the ubiquity of error. Similarly, the empirical sciences employ the machinery of hypothesis testing, controlled experiments, and the disciplined reporting of data, material, and methods in a standard format in published research articles. In the computational sciences Jon Claerbout, professor emeritus of geophysics at Stanford University, pioneered a system for linking code and data with the final paper or thesis and giving the reader the capability to regenerate the results. His insight is as follows, "an article about computational science in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures" [2] . In this way, the code and data that produced the computational results are available alongside the published paper, enabling readers to verify and validate the results. In his research lab Claerbout implemented reproducible research through the publication of reproducible figures in doctoral dissertations. The efforts pioneered by Claerbout were 978-1-4673-4466-1/12/$31.00 ©2012 IEEE
RunMyCode.org REF is cloud-based tool to expedite the process of reproducing results.
RunMyCode.org: A novel dissemination and collaboration platform for executing published computational results
2012 IEEE 8th International Conference on E-Science
2012 IEEE 8th International Conference on E-Science
Computer Science