dataesr/scientific-paragraphs-categorization huggingface dataset:
In this paper, we review and expand some of these best practices in the context of two case studies: computational magnetism and mathematics. This is based on the experience of enabling and applying Jupyter environments in these fields as a part of the OpenDreamKit (https://opendreamkit. org/) project. To be able to run computational studies from the Jupyter environment, it is necessary to either have the simulation and/or analysis code exposed to a general-purpose programming language supported by Jupyter, or have a dedicated Jupyter kernel for the computational libraries. Although the main topic of this work is the overview of features and capabilities of the Jupyter environment for reproducible workflows, we begin by discussing how a computational library can be exposed to Jupyter as a necessary prerequisite.
Prerequisite: Exposing computational libraries to the Jupyter environment Computational studies often use existing computational (legacy) tools. These could be executables called from the command line or libraries that are used within a programming language. For the approach suggested here, these computational tools need to be accessible to scientists from a general-purpose programming language supported by Jupyter (such as Python). For some domains, such as pure mathematics research, there are domain-specific languages with enough power to be used directly as the programming language in notebooks (e. g. , Singular and GAP). In other areas, exposing computational tools to a generalpurpose programming language is the key to integrating them into researchers' custom code. A key benefit of making computational tools available in a general-purpose programming language is that the computation can be driven flexibly using the control structures provided by that language.
For example, a simulation can conveniently be repeated with a range of parameters through a forloop, rather than having to change a configuration file for each value and trigger execution of the simulation manually. Making the computational capability accessible from a general-purpose programming language supported through a Jupyter kernel such as Python may be trivial -for example, if the required code is already a Python library. When the computational functionality is locked into an executable, one can create an interface layer so that functionality can be accessed via a Python function or class  [1] : input parameters will then be translated into configuration files, the executable called, outputs retrieved, and finally, the results returned. If the computational tool uses a programming language that Jupyter does not support, another possibility is to implement a Jupyter kernel for that language so that the computational library can be exposed to the Jupyter environment (as done for GAP and SageMath for example).
Features of the Jupyter research environment Project Jupyter is a set of open-source software projects for interactive and exploratory computing emerging from IPython. The central component offered by Jupyter is the Jupyter Notebook -a web-based interactive computing platform. It allows users to create data-and code-driven narratives that combine live (re-executable) code, equations, narrative text, interactive dashboards, and other rich media. Jupyter Notebook documents provide a complete and executable record of a computation that can be shared with others in a way that has not been possible before  [3]. Within the Jupyter Notebook, all libraries available in Python can be imported and combined flexibly. Other languages (such as Julia, R, Haskell, Bash, and many more) are supported through other Jupyter Notebook kernels.
One study -one document The notebook allows us to carry out an entire study within a single notebook and provides a complete and executable record of the process. It is possible to put the interpretation of the results into the same document, immediately below the graphical, tabular or text-based output that needs to be described. The "one study -one document" approach has immediate advantages: • Scientists can be more efficient as they do not have to search for parts of the study (scripts, data files, plots) when trying to understand the data and authoring the associated paper. • The study is more easily reproducible (see item 6, below). However, putting all the code, data, and narrative into a single notebook could substantially affect the notebook's readability. Thus, it is necessary to decide which parts of the code should be in libraries and imported in the notebook.
Interactive execution or as batch job Using a Jupyter notebook often involves interactively editing it, executing cells, inspecting computed outputs, modifying commands, and reexecuting, while understanding the computational research question. Once a useful processing sequence has been found, the researcher often wants to repeat that, potentially with different input data. For such scenarios, a notebook can be executed from the command line (using the nbconvert tool), treating the notebook like a script or a batch job. As the notebook executes in batch mode, it computes the output cells, including images and other multimedia, as if it were executed interactively, and the outputs are stored into the notebook file for later analysis and inspection. Execution of notebooks as a script is a convenient way to use the computational power of a high-performance computing facility where such notebook jobs can be submitted to the batch queue.
Static and interactive software documention Writing research software documentation is a particular challenge in academia. Small teams may not see the need to document their research code, as they can learn about it directly from one another. Jupyter notebooks offer an efficient method for creating documentation. The popular Sphinx documentation software can use Jupyter notebooks as the documentation source with the nbsphinx plugin, and create HTML and PDF documents. Demos and tutorials written in notebooks can complement reference documentation in Sphinx's default reStructuredText input format.
Notebooks have several benefits for extended examples in documentation: • It takes less time to create documentation as the author can type commands and explanations into the same document, and the outputs that the commands produce (text and images) appear immediately in the notebook. • After changing the user interface or computational algorithms, re-executing the documentation notebooks will often show where the documentation needs changing. • Tools like nbval can automatically re-execute the notebooks and raise test errors if the execution fails, or the computed outputs have changed. This means continuous integration can be used to check the documentation and warn developers if changes in the code affect the illustrated behaviour. • Using Binder, the documentation notebook can be executed interactively by the user (see item 5, below).
Executable interactive documents in the cloud (Binder) The open-source Binder project  [4]  and Binder instances such as myBinder offer customized computational environments in the cloud on-demand, in which notebooks can be executed interactively. To use the free myBinder service, one needs to create a publicly readable git-repository containing Jupyter notebooks and a specification of the software required to execute these notebooks. This specification follows existing standards, such as a Python-style requirements. txt file, conda environment. yml file, or Dockerfile. The myBinder service is invoked when a URL is requested containing the path to the GitHub repository. The myBinder service searches that repository for the software specification, creates a suitable container, adds a Jupyter server to the container, and exposes that server to the user.
• Providing interactive documentation: Given Binder-compatible specifications, documentation can be presented as an executable notebook through myBinder, allowing the person reading the documentation to interactively explore the software's behavior, for example by repeatedly modifying and executing commands provided as part of the documentation. • Demonstrating and disseminating small computational studies: Jupyter notebooks can be used to document computational processes. For example, for dissemination or to demonstrate reproducibility, as we explain in item 6, below. The related Voilà project can execute note- books (for example on myBinder) and hide all code cells, making an interactive dashboard to display and explore data without the source code. 6. Reproducibility -combining data, code, and software environment Reproducibility of scientific results is a cornerstone of our interpretation of science: only results that can be reproduced are accepted as proven insight.
We see an emerging trend that journals and research councils increasingly (and justifiably) ask for details on how published results can be reproduced, or at least expect authors to provide that information if a reader requests. It is often impossible to truly document an entire computational workflow, software requirements, hardware used, and other parameters within a conventional manuscript submission. The Jupyter-based research environment can help because it makes the process of publishing repro-ducible computational results easily achievable: • The "one document -one study" model automatically records all parameters, processing commands, and outputs, demonstrating the process leading to the result obtained with that notebook. By sharing the notebooks in a public repository, a DOI can be assigned via Zenodo to preserve the repository's content permanently and make it citable.
• Notebooks that create central figures and statements of publications will likely need underlying libraries. To re-execute the notebook, we need a way to specify a computational environment containing these libraries and Binder provides that possibility. Although specifying exact versions of underlying libraries is recommended, Binder does not guarantee that this would lead to the same computational environment at any point in the future, and therefore, it cannot entirely address the issue of so-called software collapse where the underlying libraries and interfaces become deprecated, compilers and compiler optimization methods change, etc. • By publishing the notebooks reproducing central results together with software environment specifications for Binder in an open repository, anyone with Internet access and a browser can inspect and re-execute these notebooks and thus reproduce the publication.
Remote access to institutional compute resources -JupyterHub The discussion above assumed that notebooks were running on the user's computer. The Jupyter-Hub software allows institutional provision of Jupyter Notebook services. It allows users of an institution to authenticate with their organizational credentials and access a Jupyter environment running on the institution's infrastructure. Typically, any files and folders the user is allowed to access will also be made available to them through JupyterHub, including access to shared data and folders where they can save their notebooks. The institution generally predefines the soft-ware environment in which the notebook server executes. However, the technology is available to use the software specification as for Binder to create a customized computing environment ondemand.
A vital point of the user experience is that only a web browser is required to access the JupyterHub and to carry out computational work using these resources remotely. Figure  2  shows an artistic illustration of the scenario where an instructor works with their institution to provide students with a customized software environment. Other use cases of JupyterHub installations include research facilities and universities providing access to their (high-performance) computing resources through Jupyter notebooks, where traditionally ssh or remote desktops may have been used.
Blending script and GUI-driven exploration methods The IPyWidgets Jupyter extension provides selection menus, sliders, radio buttons, and other GUI-like graphical interaction widgets to Jupyter notebooks. The Notebook allows embedding such graphical widgets inside the notebook, and users can combine the usual scripted analysis with activation of such widgets where desired. They can be used, for example, to vary the input parameter values and explore a data set or computational results. Although less reproducible than typed commands, widgets can be useful for rapid feedback on different possibilities.
Potential disadvantages Above we focused on the features and capabilities of the Jupyter research environment to support computational workflows in science. Here, we want to discuss some downsides that have come up either in our work or as feedback from users of Jupyter-based computational tools we developed. (a) Undefined notebook state Top-to-bottom arrangement of cells in a notebook implies that they should be executed in that order. One of the Jupyter Notebook's key features is that the code cells can be executed in an arbitrary order -the user can select (and modify) any cell and then execute it. This can be useful while exploring a data set or a property of computation, or even to debug the cell's code. The execution order used in a notebook is not stored when the notebook is saved. Therefore, it is critical to remember that, by executing cells out of order, we may create different results from when we execute all cells in order.
Besides, postprocessing and analysis of results is performed outside OOMMF, using techniques and scripts that are mostly developed by the user, or carried out manually. Consequently, it is hard to track, record, and convey the exact simulation procedure. Without this information, resulting publications are generally not reproducible. To address this situation, we developed a Python interface to the OOMMF executable. This allows us to conduct computational magnetism simulations from within the Jupyter notebook to capitalize on the benefits of this environment. We developed a set of Python libraries we refer to as Ubermag, which expose the computational capabilities of OOMMF so that it can be controlled from Python. These Python libraries provide a domain-specific language to define a micromagnetic problem  [1].
A micromagnetic model, defined using the domain-specific language, is not aware of the particular simulation tool that will perform the actual micromagnetic simulation, and it is only used to describe the model. When a simulation is required, the model is translated into the OOMMF configuration file, the OOMMF executable is called, and the output files are read. By exposing the micromagnetic simulation capabilities to Python and driving the research from Jupyter Notebook, we have available all the benefits of the Jupyter research environment. To demonstrate the use of Ubermag, we use standard problem 3 as an example. Standard problem 3 is a standardized problem posed by the micromagnetic community to test, validate, and compare different simulation tools. It describes a magnetic cube of edge length L with two different magnetization states that can occur as local energy minima, called the flower state and the vortex state.
Beg et al. Using Jupyter for reproducible scientific workflows. GitHub: https://gith ub. com/marijanbeg/2021-paper-jupyter-reprodu cible-workflows, DOI: 10. 5281/zenodo. 4382225 (2021)]. We show the two most relevant code cells inside the Jupyter notebook in Figure  3. Ubermag and the Jupyter environment simplify the efforts to make computational magnetism publications reproducible. For each figure in the publication, one notebook can be provided (find examples in Refs. [6] ,  [7] ). Using Binder, the community can inspect and re-run all the calculations in the cloud and make the publication reproducible.
Computational studies in mathematical research Many of the leading open-source mathematical software systems (including GAP, LinBox, PARI/GP, OSCAR, SageMath, and Singular) have been made inter-operable with the Jupyter ecosystem through bespoke or general-purpose kernels (C++, Python, Julia,. ). Focusing on one of these systems, for the sake of concreteness, we illustrate how this supports sharing and publishing reproducible computational studies in mathematical research together with the underlying research code. GAP is an open-source system for discrete computational algebra, with particular emphasis on computational group theory. It is used routinely by mathematicians in these fields and beyond to support teaching and research, notably through computational exploration. It provides a domain-specific language, also called GAP, and a runtime system with a command-line interface.
It can also be used as a library by other systems such as SageMath or OSCAR. GAP has been developed for decades by a community of researchers, teachers, and research software engineers. It has an established mechanism for user-contributed extensions, called packages, which may be submitted for the redistribution with the system, and a formal refereeing process. The current release of GAP (4. 11. 0) includes 152 packages that serve different purposes, from providing data libraries and extending the system's infrastructure for testing and writing documentation, to adding new functionality and sharing research codes that underpin their authors' publications. The latter scenario may require specific expertise and motivation from a working mathematician who uses GAP, and not everyone will be able to invest efforts in sharing their code in this way.
Furthermore, it is not always justifiable to organize a supplementary code for a paper as a new GAP package. Instead, authors can combine Jupyter research environments with additional services and parts of the infrastructure for GAP packages to share reproducible computational studies while following good code development practices from the start. Let us illustrate this with the publication in Ref. [8] , which presents a polynomial-time algorithm for solving a major problem in computational group theory, which remained open since 1999  [9]. An essential addition to the paper is the author's GAP implementation of the algorithm. The authors published this implementation in the publicly hosted repository. At once, this ensures long term archival through the Software Heritage project, and with a small additional step, it makes it citable through Zenodo.
The repository contains an interactive narrative document -a Jupyter notebook using the GAP Jupyter kernel  [10]  combining text, mathematics, inputs, and outputs, and may even be viewed as a slideshow (one could, of course, have separate notebooks for different purposes). Following best practices for organizing reproducible computational studies (see e. g. , Ref. [11] ), the code is not written in the notebook itself but loaded from external source files. These are text files that can be easily managed with version control, reused from multiple Jupyter notebooks, and tested using the GAP automated testing setup. Also, the authors made the repository Binder-ready. Any user (e. g. , readers or referees of the paper) can run the notebook and reproduce its execution on Binder itself or -with additional expertise to install the required assetson their own computing resource.
To achieve this, the authors followed the template in Ref. [12] , which also brings in continuous integration to automatically check the code against several past, current, and development releases of GAP, and produce coverage reports on how thoroughly the tests exercise the code. It boils down to creating a tst directory with the test files, and adapting the configuration files. travis. yml and. codecov. yml for Travis CI and Codecov services, respectively. Bringing Jupyter interfaces to command-line based computational mathematics tools makes it possible to interface it with numerous JavaScript libraries, notably for visualization. For example, GAP packages Francy and JupyterViz extend the GAP Jupyter kernel  [10]  with interactive widgets and plotting tools, which can be tried from their Binder-ready repositories.
Introduction Continual Learning (CL), also referred to as Lifelong or Incremental Learning, is a challenging research problem  [7]. Lately, it has become the object of fast-growing interest from the research community, especially thanks to recent investigations leveraging gradient-based deep architectures  [15, 37]. In the last few years, machine learning has witnessed a prolific, variegated, and original research production on the topic: from Computer Vision  [9, 30, 36]  Figure  1 : Operational representation of Avalanche with its main modules (top), the main object instances (middle) and the generated stream of data (bottom). A Benchmark generates a stream of experiences e i which are sequentially accessible by the continual learning algorithm A CL with its internal model M. The Evaluator object directly interacting with the algorithm provides a unified interface to control and compute several performance metrics (p i ), delegating results logging to the Logger(s) objects.
We designed Avalanche according to a set of fundamental design principles (Sec. 2) which, we believe, can help researchers and practitioners in a number of ways: i) Write less code, prototype faster, and reduce errors; ii) Improve reproducibility; iii) Favor modularity and reusability; iv) Increase code efficiency, scalability and portability; v) Foster impact and usability. The contributions of this paper can be summarized as follows: 1. We propose a general continual learning framework that provides the conceptual foundation for Avalanche (Sec. 3). 2. We discuss the general design of the library based on five main modules: Benchmarks, Training, Evaluation, Models, and Logging (Sec. 4). 3. We release the open-source, collaboratively maintained project at https : / / github. com / ContinualAI/avalanche, as the result of a collaboration involving over 15 organizations across Europe, United States, and China.
Comprehensiveness and consistency The main design principle for Avalanche follows from the concept of comprehensiveness, the idea of providing an exhaustive and unifying library with end-to-end support for continual learning research and development. A comprehensive codebase does not only provide a unique and clear access point to researchers and practitioners working on the topic, but also favors consistency across the library, with a coherent and easy interaction across modules and sub-modules. Last but not least, it promotes the consolidation of a large community able to provide better support for the library. Ease-of-Use The second principle is the focus on simplicity: simple solutions to complex problems and a simple usage of the library. We concentrate our efforts on the design of an intuitive Application Programming Interface (API), an official website, and rich documentation with a curated list of executable notebooks and examples  1.
This is essential for scalability and to collaboratively bring the codebase to maturity. A particular focus on module independence is maintained to guarantee the stand-alone usability of individual module functionalities and facilitates learning of a particular tool. Efficiency and Scalability Computational and memory requirements in machine learning have grown significantly throughout the last two decades  [52]. Standard deep learning libraries such as TensorFlow  [1]  or PyTorch  [39]  already focus on efficiency and scalability as two fundamental designing principles, since modern research experiments can take months to complete  [48]. Avalanche is based on the same principles: offering the end-user a seamless and transparent experience regardless of the use-case or the hardware platform that the library is run on.
Main Modules The library is organized into five main modules: Benchmarks (Sec. 4. 1), Training (Sec. 4. 2), Evaluation (Sec. 4. 3), Models (Sec. 4. 5), and Logging (Sec. 4. 4). Table  1  summarizes the features provided by Avalanche at the current stage of development. In Fig. 1 , an operational representation of the library modules and their interplay within the aforementioned reference framework is shown.