dataesr/scientific-paragraphs-categorization huggingface dataset:
The possibility of packaging code and data together has turned out to be very useful. In deciding how to divide the complete information resulting from the research projects into ActivePapers, the major considerations were the following: • What makes a useful package for reuse? • Which information must be consulted together for understanding the computational methods? • Which items are required to back up the claims of a journal publication? The application of these criteria led to three kinds of packages: • Software libraries. • Datasets packaged with the software that produced them, and sometimes also with software that facilitates access to or analysis of the data. • "Supplementary material" packages for a journal article, containing the top-level workflow up to the generation of all figures and tables.
We did not encounter any situation where it would have been useful to publish datasets without any accompanying software. For all datasets, there was some software that logically belonged to the data as an aid in its interpretation. However, this observation may be related to the fact that all the computational studies performed within the ActivePapers framework were of methodological nature, i. e. contained the development of new computational methods. As an example, reference 68 is a database of partial protein structures. They were constructed by downloading complete protein structures from the Protein Data Bank 1 and selecting parts that have specific structural features. The ActivePaper file includes the script that performs the download and the substructure extraction. This script is a detailed documentation of the selection criteria, and therefore valuable for anyone wishing to use the database.
One problem encountered in the course of these research projects is the relatively low size limit that today's digital repositories impose on archived files. Zenodo 5 provides the most generous limit of 2 GB per file. However, the input data for one study 71 contains ten Molecular Dynamics (MD) trajectories for lysozyme in solvent, and requires 10 GB of storage even in compressed form. Since these data were not essential for the subsequent analysis step, which requires only the rigid-body motion of the protein, they were removed from the published files. The alternative would have been to publish each MD trajectory separately as an ActivePaper, and use DOI-based references in the analysis step to refer to this data. The main drawback to this approach is the additional effort required for dividing the data into small packages and publishing them separately.
ActivePapers proposes another mechanism to reduce file sizes: the deletion of recomputable datasets. Any dataset that was generated by a program stored in an ActivePaper can be replaced by a dummy dataset that retains only the dependency metadata. The full dataset can be recomputed on demand, or automatically when another program tries to read it. Recomputation consists in re-running the program that generated the dataset initially. This mechanism makes sense only if the replicability of a dataset is guaranteed. In practice, this applies to any program that does not use floating-point operations and whose input data is read exclusively from the ActivePapers universe. The ActivePapers implementation tests the second condition but cannot verify the first.
Future developments The existence of two distinct ActivePapers platforms is a historical accident that is clearly not desirable. The envisaged solution is a split of the ActivePapers platform definition into two parts: a data publishing system, which defines the HDF5 conventions for Active-Papers, in particular the metadata, and a code execution system that defines how specific datasets in ActivePapers are interpreted as executable code. Only the second part would differ between the current two implementations, and its separation also opens the way for additional execution systems for other code representations. One line of future development is an integration of a narrative into the computational methods stored in an ActivePaper. Work on integrating the ActivePapers Python edition with the Jupyter project 30 (formerly the IPython notebook) is underway.
The second solution is to record all interactive code execution in a log, which is stored in the ActivePaper and can be re-executed later. After a complete linear execution of the notebook code, the log of interactive executions can be deleted. Another direction for future developments explores how to provide a realistic transition from today's scientific computing environments to future ones that take into account the needs for publishing and archiving computations. One important advantage of the ActivePapers approach in this context is that the minimal requirements for adopting it are modest: any software tool that can work with the ActivePapers file format, which is HDF5 plus a small set of conventions, can read and write publishable datasets. With a very small additional effort, software tools can be adapted to handle ActivePapers metadata and thus ensure dependency and provenance tracking.
Conclusion Experience with the two current implementations of the ActivePapers idea has shown that all of the requirements defined at the outset can be fulfilled and that the approach of publishing data and code packages works in practice. In particular, the ActivePapers project has shown that installation-free software deployment and long-time software preservation are possible, contrary to a common belief in the scientific computing community. ActivePapers can achieve these goals because the scientific contents of software take the form of pure computations, which are possible identically in all computational environments. The technological basis is a precise and stable definition for a computational platform, which includes a representation for executable code and a runtime library for interaction with the computational environment.
Experience with ActivePapers in real research projects has led to further important observations: • Once software has a well-defined representation, it becomes a piece of data that can and should be handled exactly like any other scientific dataset. • Packaging data and software together is natural in many situations because of the close relation between them. • Installation-free deployment turns replication into a mechanical procedure that can be automated. • Reuse of published data, including software, is the computational equivalent of a citation. It can be implemented using technology developed for citation (DOIs), and thus easily integrated into science metrics. However, the ActivePapers experience has also shown that the majority of today's scientific software does not fulfill the conditions for inclusion in the ActivePapers framework, nor in any similar framework based on the same design goals.
For instance, this approach provides portability across platforms and bytecode written for a JVM has remained executable for decades, while compromising on details such as architecture-specific nature of performing floating-point computations. The author seeks to address this in part through an alternative Python implementation, but laments that external library dependencies required to do research in Python and the difficulties in tracking strong versioning available in the bytecode implementation. While I appreciate the author raising this often-overlooked issue of external library dependencies, which impacts Python, R, and many commonly used research languages, I worry that the paper may be overstating the concern. For one, the author does not present any evidence as to how often changes in these system dependencies really impact the reproducibilty of the code.
such as containerization (as implemented by Docker but also by others in the scientific context, see the F1000Research such as containerization (as implemented by Docker but also by others in the scientific context, see the approach taken at CERN: ). Understandably this paper need not review all http://arxiv. org/abs/1407. 3063 other technology, but where it does so it would be useful to map these more clearly to the four criteria above. What criteria are being met by the other approaches the author has considered? What are being only partially met? What are missing? The current section "Tools for reproducible research" is too cursory an overview and is not tied back to the criteria or strengths/weaknesses of the ActivePapers approach. I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
A concrete computational study may not require that much, but any "close enough" criterion is necessarily application-specific. Interestingly, most of today's computational platforms do not make any promise about floating-point reproducibility, creating the same problem as with external libraries: the reproducibility of my code relies on something that is completely out of my control. Point 3, security issues, and point 4, metadata, do indeed require a better treatment in the paper. In short, the JVM edition has good security support (inherited from the JVM libraries), whereas the Python edition has almost none, because the Python platform has none. Metadata handled by the ActivePapers platform will be documented in detail in the revision. Note that users can add any metadata they like, following any standards they consider useful.
The metadata defined by the platform is limited to what can be automatically generated from the history of an ActivePaper. This article has been thoroughly revised. Its purpose is clear, and its finding are supported by a mixture of literature and software review and experience with ActivePapers. We have two minor suggested revisions. First, the four bullets in the conclusions could also be worked into the introduction. We see these as well-stated, important points to make early in the paper. For example, the first bullet---thinking of a software representation as a dataset and archiving it accordingly---is a simple, intuitive way to think about much of what is discussed in this paper. Second, on page 4, the author says "many practitioners consider the idea of preserving scientific software for many years unrealistic, and some even argue that it is unnecessary because computational methods change so rapidly that their long-term preservation is of no interest.
" This statement needs citations. We have read this submission. We believe that we have an appropriate level of expertise to to make a dataset reusable in the longer term: First, researchers are encouraged to use and share datasets in formats commonly used by their discipline, and when possible, formats that do not depend on proprietary software. When the data file format depends on a specific software (or is not considered a preservation format), there exist software tools that convert the proprietary format to a preservation format. For example, for tabular data in SPSS and STATA format which depend on a specific statistical package, the files can be converted to a plain text (tab-delimited or CSV file) plus a metadata file (in XML or JSON format) that contains information about the columns in the original tabular data file.
Information is not lost and it can be re-combine to generate an SPSS or STATA file. This preservation feature can be found in public data repositories such as the Dataverse repository software ( ). There are software projects that focus on other similar automated preservation dataverse. org tools to re-format data files into preservation formats and provide additional preservation metadata. One example is Archivematica. These preservation tools are important for data repositories if they want to make their data accessible and reusable in the future, when the original software might be obsolete. On Methods/Code/Software One approach on sharing the code used to model/analyze a research work is by using an open-source language like R, where the models and packages are shared and disseminated through a common R package repository (CRAN).
Indexability This article requires considerable work to be indexed. First, clarify the type of article that is being written, and restructure accordingly. Second, the author misses some key existing technologies and practices that should be discussed. We have read this submission. We believe that we have an appropriate level of expertise to This article by Konrad Hinsen discusses the very important issue of how we capture the detail of the dependencies and environment surrounding the software tools that we use, such that we can guarantee that the tools may be used in the future to replicate research and reuse the tools. Overall, it is a comprehensive summary of most of the area. However as it stands, I believe that the article could be significantly improved by deciding whether the paper is to be a fully comprehensive "state of the art" summary, looking at the specific advantages and disadvantages of each approach; or one which sets out a shorter summary of the state of the art (as is present in the article in this version), and then goes on to describe the lessons learned from the ActivePapers work, in which case I would suggest a change in title to reflect the emphasis on ActivePapers as a primary example of a platform for publishing and archiving computer aided research.
Therefore I am marking this as "Approved with reservations" as it requires a structural change, rather than because I believe the work contained in it is not technically sound. In both scenarios for how the article could be rewritten, I believe the article would benefit from the use of more devices to highlight key points and comparisons, for instance by use of tables to compare the effect of different types of platforms on the ability to define environmental dependencies, or linkage with input data. There is one important area of research that should be covered in the discussion of the state of the artthat around "significant properties" of software. In particular, the work of Brian Matthews at STFC in this area has previously considered the issue of capturing and prioritising details of both the environment, and how a computational tool is expected to function, thus forming a theoretical basis for describing the way that many current implementations ranging from dependency managers like Maven, through configuration management tools like Docker, Vagrant and Conda, to virtualisation.
See: http://www. jisc. ac. uk/media/documents/programmes/preservation/spsoftware_report_redacted. pdf http://ijdc. net/index. php/ijdc/article/view/148 In terms of other areas where I felt that additional discussion would have provoked more debate, these would be around the trade-offs surround floating point operations, a discussion of other bytecode platforms, and around the long lifetime of successful pieces of software, in particular around trust and how it is mechanically/technically checked. This last point is illustrated in this example from random sampling: https://cryptogenomicon. wordpress. com/2014/10/13/cryptic-genetic-variation-in-software-hunting-a-buffered-41-year-o As minor points that I believe would improve the papers I would suggest the following: The term "Web repositories" for platforms like FigShare and Zenodo is not commonly usedindeed, it is more commonly used to refer to repositories of web pages.
I would suggest the more commonly used "digital repositories" term, or perhaps "web accessible third party digital repositories"? On page 3, it would be useful to describe what makes "Web repositories" better. I would suggest it is cost (most are free for openly licensed deposits) and the ability to generate citable DOI s On page 3, when talking about the versioning used by , it should be clarified that "a F1000Research F1000Research On page 3, when talking about the versioning used by , it should be clarified that "a F1000Research DOI refers permanently to a specific version of an article" On page 3, the author's example of the DSSP method still being widely used today could be seen to contradict the earlier arguments surrounding the inability of code to preserved effectively. Whilst I agree that as stated on Page 5 "This lack of a precisely defined and stable platform for executable code is also the root cause of non-replicability in computer-aided research" I feel that the author could discuss the tradeoffs (mostly optimisation and performance based) in more detail and put across their opinion of which are most important.
On page 6, I had a little difficulty with the statement that "the scientific aspects of software are always pure computation". I think that I understand what the author means, but as written it makes me want to identify a counter example. In fairness, I haven't been able to find one yet. On page 7, I note that all the examples of ActivePapers I could find on FigShare have the author as a co-author on them. This means it is unclear whether ActivePapers Python edition is indeed suitable for the wide variety of scientific research areas, as it is unclear whether they represent a representative selection of the use cases for ActivePapers. I believe that the author could do more to support their statement that the fundamental distinction is between "computational tool" and "scientific content".
The author and I have both participated in the "Code as a Research Object" Competing Interests: community which is referenced in this article. I am a co-organiser of the WSSSPE community which is mentioned in the article. Author Response 24 Feb 2015 , Centre de Biophysique Moléculaire (CNRS), France Konrad Hinsen My goal was not to write a review article on the topic of computational reproducibility, but to present the conclusions from developing ActivePapers and applying it in real-life research projects. The article has been restructured accordingly. The review of prior work has been shortened and reframed in the context of the ActivePapers project. The "lessons learned" from ActivePapers are presented in more detail. I was not aware of the work of Brian Matthews on software preservation in the context of scientific I was not aware of the work of Brian Matthews on software preservation in the context of scientific computing.
Computational science:. Error. Nature. 2010; 467(7317): 775-777. PubMed Abstract | Publisher Full Text 8 PubMed Abstract | Publisher Full Text 2. The Dataverse project. 2015. Reference Source 3. Dryad. 2014. Reference Source 4. figshare. 2014. Reference Source 5. Zenodo. 2014. Reference Source 6. Academic torrents. 2014. Reference Source 7. Merali Z:. Joppa LN, McInerny G, Harper R, et al. : Computational.
J Chem Phys. 2013; 139(12): 124115. PubMed Abstract | Publisher Full Text 64. Chevrot G, Hinsen K, Kneller GR: Model-free simulation approach to molecular diffusion tensors. J Chem Phys. 2013; 139(15): 154110. PubMed Abstract | Publisher Full Text 65. Hinsen K: MOSAIC: a data model and file formats for molecular simulations. J Chem Inf Model. 2014; 54(1): 131-137. PubMed Abstract | Publisher Full Text 66. Hinsen K: ImmutablePy 0. 1 in ActivePapers format. figshare. 2013. Reference Source 67. Hinsen K: pyMosaic 0. 3. 1. Zenodo. 2014. Reference Source 68. Hinsen K: ASTRAL-SCOPe subset 2. 04 in ActivePapers format. Zenodo. 2014. Reference Source 69. Hinsen K, Shuangwei Hu, Kneller GR, et al. : A comparison of reduced coordinate sets for describing protein structure. figshare. 2013. Reference Source 70.
Chevrot G, Hinsen K, Kneller GR: Model-free simulation approach to molecular diffusion tensors: Water. figshare. 2013. Reference Source 71. Chevrot G, Hinsen K, Kneller GR: Model-free simulation approach to molecular diffusion tensors: Lysozyme. figshare. 2013. Reference Source 72. Hinsen K: pyMosaic 0. 1. 1 in ActivePapers format. figshare. 2013. Reference Source 73. Hinsen K: pyMosaic 0. 2. 0. Zenodo. 2014. Reference Source 74. Hinsen K: pyMosaic 0. 3. 0. Zenodo. 2014. Reference Source 75. Nested VM - binary translation for Java. 2015. Reference Source 76. Katz DS, Choi ST, Lapp H, et al. : Summary.
The Challenge on Liver Ultrasound Tracking (CLUST) was held in conjunction with the MICCAI 2014 conference to enable direct comparison of tracking methods for this application. This paper reports the outcome of this challenge, including setup, methods, results and experiences. The database included 54 2D and 3D sequences of the liver of healthy volunteers and tumor patients under free breathing. Participants had to provide the tracking results of 90% of the data (test set) for pre-defined point-landmarks (healthy volunteers) or for tumor segmentations (patient data). In this paper we compare the best.
g. vessels) are tracked and these tracking results are used as input surrogate data to 4D motion models to predict the tumor position  (Tanner et al 2012 , McClelland et al 2013). Despite the rapid development of image-guided therapy, intervention systems and medical imaging tools, the translation into clinical practice of automated motion estimation is limited. One of the main reasons for algorithms not being integrated in clinical practice is the lack of adequate validation. Open datasets for designing and testing tracking algorithms are missing, and private datasets differ in size, image dimension and sequence length. The variations in tracking objective (full organ, anatomical landmarks, tumor) and validation strategies are additional impediments to strategy comparisons. For image-guided therapies, tracking methods should have high accuracy, robustness over the duration of the therapy and real-time capability.
Several methods have been proposed for tracking human liver structures on US sequences. Yet quantitative evaluation of tracking the human liver under free breathing was reported only by  Banerjee et al (2015) ,  Harris et al (2010) ,  Lediju Bell et al (2012)  and  Vijayan et al (2014)  for 3D sequences and by  Cifor et al (2012 Cifor et al ( ), (2013)) , De Luca et al  (2012)  and De Luca et al  (2013)  for 2D sequences. Intensity-based and hybrid approaches achieved good accuracy (∼1. 4 mm mean tracking error  (Harris et al 2010)  and ∼90% mean overlap ratio  (Cifor et al 2013) ). The non-rigid registration method of Vijayan et al (2014) estimated liver motion with an error of 1 mm (75% percentile of a root-mean squared metric over all datasets), which was lower than the inter-observer variability of 1.
Ultrasound data The collected database included a total of 54 US sequences of the liver of patients and volunteers under free breathing. The data were provided by 6 groups, namely the Computer Vision Laboratory, ETH Zurich, Switzerland (ETH)  (De Luca et al 2013 , Preiswerk et al 2014) ; mediri GmbH, Heidelberg, Germany (MED); Institute of Biomedical Engineering, University of Oxford, UK (OX)  (Cifor et al 2013) ; Biomedical Imaging Group, Departments of Radiology and Medical Informatics, Erasmus MC, Rotterdam, The Netherlands (EMC)  (Banerjee et al 2014) ; Joint Department of Physics, Institute of Cancer Research & Royal Marsden NHS Foundation Trust, London and Sutton, UK (ICR)  (Lediju et al 2010 , Lediju Bell et al 2012) ; and SINTEF Medical Technology, Image Guided Therapy, Trondheim, Norway (SMT)  (Vijayan et al 2013).
The sequences were acquired with 6 US scanners, 7 types of transducer and different acquisition settings. An overview of the data is given in table 1. The length of the sequences ranges from 4 s to 10 min, with a temporal resolution in the range of 6-25 Hz. The dataset is divided into three subsets, according to the image dimension and annotation type. The first subset is composed of 28 2D sequences from healthy volunteers with point-landmark annotations. The second subset contains 10 2D sequences from 5 patients with segmentation annotations. The third subset consists of 16 3D sequences with point-landmark annotations from healthy volunteers. The data were anonymized and divided into a training set (10% of the sequences) and a test set (90%). For the training set annotations were released, to allow for some tuning of the tracking    4V-D.