dataesr/scientific-paragraphs-categorization huggingface dataset:
3D Alignment and Reconstruction We introduced a gridding-based direct Fourier method for the three-dimensional reconstruction approach that uses a weighting technique to compute a uniform sampled Fourier transform  [22]  in 2015. In 2019, we accelerated this algorithm  [23]  as part of the extended collaboration with the High-Performance Computing research group at the CERIT-SC Centre in the Czech Republic. Both the CPU and GPU versions are available via the Reconstruct Fourier protocol. While participating in the Map Challenge by the Electron Microscopy Data Bank, we developed the High-Resolution Reconstruction Protocol (HighRes)  [24]. This protocol uses an approach similar to the standard projection matching with some important modifications, especially in detecting significant features in the reconstructed volume.
Model Building Partially related to model building is our contribution to the 3D model construction from the atomic structures using a very accurate conversion with Electron Atomic Scattering Factors  [44]. It is available via the Convert PDB protocol. In 2020, we contributed towards the inter-package integration of the model-building tools in Scipion [45] by adding several protocols, e. g. , Extract Asymmetric Unit protocol or Export to DB protocol to help in the export process to the EMDB/PDB database. Note that to see these protocols, Scipion View has to be changed to the Model building. To evaluate the quality of the map-to-model fit, we have proposed the FSC-Q measure  [46]  available via the Validate FSC-Q protocol, which is a quantitative estimation of how much of the model is supported by the signal content of the map.
In 2014, we proposed a standard for transferring the information on the three-dimensional orientation between packages  [57]. In 2017, we provided a detailed survey of the iterative reconstruction algorithms used in SPA and Electron Tomography  [58]. In the same year, we also analyzed theoretical foundations and derivation of several concepts and thresholds used for resolution assessment in 3DEM  [59]. In 2019, we published a survey of the analysis of continuous conformational variability of biological macromolecules [60] and reference analysis of the β-galactosidase using streaming in Scipion  [61]. In 2020, we showed that global B-factor sharpening and deposition of only the sharpened maps in the Electron Microscopy DataBase could be detrimental  [62]. In the same year, we also published a review of local resolution concepts and algorithms  [63].
GPU Acceleration Several of the Xmipp protocols and programs have their computationally intensive portions of the code accelerated via GPU using the CUDA Toolkit. The deep learning programs then use TensorFlow or Keras. The list includes the most performance critical protocols, such as CL2D (GL2D)  [15] , DeepAlign  [27] , RANSAC  [16] , FlexAlign  [6] , Projection Matching, Reconstruct Fourier  [23] , Reconstruct Significant  [18] , HighRes  [24] , Swarm Consensus  [19] , Split Volume, and Validate Overfitting protocol. Programs using deep learning and the Optical Flow movie alignment can be executed both on CPU and GPU, though GPU is recommended for performance reasons. We also use two additional tools to further optimize the performance of the GPU code. We have experimentally used the Kernel Tuning Toolkit (KTT)  [68]  to optimize the execution of several programs on the most commonly used GPUs.
Scipion Protocol Popularity Scipion provides a list of the most used protocols at http://scipion. i2pc. es/report_ protocols/protocolTable/. Provided that the user agreed with this data collection, each time the Scipion project is opened, a list of protocols used within this project is sent to our servers. This information is useful for checking which protocols are more used than others and concentrating on any performance issue related to those. This database currently holds information about over 25,000 workflows opened since November 2016. At the time of writing this article (August 2021), Xmipp provided 37 out of the 100 most popular protocols. Out of them, the Manual|Auto Picking protocol  [10] , CL2D  [15] , HighRes  [24] , MonoRes  [38] , and several auxiliary protocols were the most used (each one has been used over 3000 times).
Technologies Used in Xmipp As mentioned before, Xmipp is a suite of programs and (Scipion) scripts. It is a collaborative open source project hosted on GitHub, divided into four main repositories: We use multiple technologies to parallelize the execution of our binaries. In addition to MPI (https://www. open-mpi. org/) and built-in parallelization in Scipion, we use the CTPL library (https://github. com/vit-vit/CTPL) for multithreading, CUDA (https://developer. nvidia. com/cuda-toolkit) and cuFFTAdvisor (https://github. com/ HiPerCoRe/cuFFTAdvisor) for GPU acceleration, and deep learning via TensorFlow (https://www. tensorflow. org/) and Keras (https://keras. io/). Experimentally, we also use StarPU (https://files. inria. fr/starpu/) for processing on heterogeneous machines and KTT (https://github.
com/HiPerCoRe/KTT) for CUDA kernel optimization. To ensure a certain quality of the code, we use a combination of unit testing via googletest (https://github. com/google/googletest), GitHub Actions for automatic project build, and static code analysis via SonarCloud (https://sonarcloud. io/organizations/i2 pc/projects), pull request reviews, and integration testing via dedicated buildbot (https: //buildbot. net/, http://scipion-test. cnb. csic. es:9980/)).
Authors acknowledge the support and the use of resources of Instruct, a Landmark ESFRI project. J. V thanks economical support from the Spanish Ministry of Science and Innovation through the call 2019 Proyectos de I+D+i-RTI Tipo A (PID2019-108850RA-I00). We acknowledge the support of the French National Research Agency-ANR (ANR-11-BSV8-010-04, ANR-19-CE11-0008-01, and ANR-20-CE11-0020-03 to S. J. ), the French National Center for Scientific Research and the Spanish National.
C++ files (419,000 LOC (Lines of Code, comments excluded, including tests)), 278 Python files (55,500 LOC), and almost 200 Java files (31,100 LOC), contributing to the 290 executables and scripts used in 110 Scipion protocols. Xmipp requires C++11 compatible compiler and JDK 11. Scipion protocols are written with Python 3. x. Xmipp provides Python binding, as well as optional Matlab binding. Optionally, Xmipp can use CUDA 8 to 11 and OpenCV versions 2 to 4. Xmipp uses SCons (https://scons. org/) as its construction tool. • Xmipp (https://github. com/I2PC/xmipp/) is the main repository. • XmippCore (https://github. com/I2PC/xmippCore/) contains code responsible for data handling. • XmippViz (https://github. com/I2PC/xmippViz/) contains code responsible for data visualization. • Scipion-em-xmipp (https://github.
Food web models, mostly Ecopath with Ecosim (EwE)  [7, 8] , have played an important role in the implementation of the EBFM. CaN modelling further contributes to this effort by providing a modelling framework for marine food-webs that primarily relies on ecological constraints which can be agreed upon by diverse actors. While the equations behind CaN modelling are simple, one major technical challenge is to sample a polytope in a highly dimensional space. This is particularly useful if achieved inside the R framework  [9] , which is widely used by ecological modelers. Until now, there existed no software to easily perform CaN modelling. The first models were constructed in an ad hoc fashion by assembling and structuring relevant data and developing tailor-made programs in Mathematica or R to run the model.
This made it difficult to communicate and replicate CaN models. It also restrained the access of the model to non-modelers. Making CaN modelling transparent, replicable and efficient in a participatory modelling framework requires that model construction, sampling and output visualization be easily accessible by modelers and non-modelers alike, that the structure of CaN models be standardised and that the model sampling be robust and effi-cient. A graphical user interface (GUI), which provides an easy entry level for users is also a highly desirable feature for a modelling approach targeting nonmodelers. In this contribution, we present a software for the implementation of CaN modelling in R, called RCaN, and an associated GUI in Java, called RCaNconstructor. RCaN and RCaNconstructor can be used jointly or separately.
Their combined use allows users to easily construct, document, sample and interpret the results of CaN food-web models. The relationship between RCaN and RCaNconstructor is shown in figure  1. The main features of RCaN and RCaNconstructor include: 1. a user-friendly interface that allow multiple users to jointly develop and revise models. Users can then focus on the discussion about their knowledge of the ecological system and how to best structure a model that fits their needs, 2. a file and data management system that ensures consistency between the different elements of a CaN model. For example, it must be guaranteed that each link connects existing components or that each constraint points to components, links or data series that are adequately specified in the model,   There are several ways to go through these stages; the ones that have been followed for the Barents Sea example (section 6) appear in dark gray in the figure.
RCaN and RCanConstructor have been designed for being part of a participatory approach: the design of the file, especially of constraints, and the interpretation of most results are simple and easy to understand. In section 2 we summarize the elements of the CaN modelling approach. In section 3, we present the format of an RCaN file. In section 4 we present the R library RCaN which is designed to construct, sample and produce graphical outputs of CaN food-web models. In section 5 we introduce RCaNconstructor, the graphical user interface that facilitates the co-construction of CaN models and the inclusion of meta-information. In section 6, we briefly describe the Barents Sea trophic network which we use as a case study example. Finally, we discuss the possible future developments of the RCaN and RCaNconstructor softwares.
Introduction RCaN is an R package to build, sample and visualize the results of CaN foodweb models. It does this by 1) constructing the polytope from implicit/explicit constraints, 2) solving the numerical challenges of polytope sampling and 3) producing graphical outputs. The next sections detail the technical features of the implementation of these different steps, in the R language. Using the constraints formula and the symbolic links among variables, RCaN automatically builds matrices C and A and vectors b and v for each active constraint. This is done by a simple call to the function buildCaN which reads the information from the RCaN-file and returns a single object that contains the trophic network and the corresponding polytope matrices. For example, in the Barents Sea case study (section 6), the function automatically constructs the matrix A which has 2593 rows and 775 columns, and the matrix C which has 54 rows (and by definition, the same number of columns).
Checking polytope characteristics At first try, the specification of model can lead to an empty polytope (i. e. , a model with no solution that satisfies all the constraints) or an unbounded polytope (i. e. , a model in which the specified constraints are not sufficient to bound a given parameter that can vary subsequently to infinity). Before running the model, it is wise to check the status of the polytope (the model won't run with an empty polytope while results are generally not relevant with unbounded polytopes). The package provides different functions to check the properties of the polytope, of which the two most important are checkPolytopeStatus, which checks whether the polytope is closed and non-empty, and getAllBound-sParam, which estimates the bounds for all parameters. If the model is empty, the RCaN function findingIncompatibleConstr can be used to identify which constraints are incompatible with each other.
Technical considerations The development of the packages raises several challenges. The first challenge was to improve computation performance. For this purpose, RCaN is interfaced with C++ using two R packages Rcpp  [11]  and RcppEigen  [12]. The second challenge was to check the characteristics of the polytope. This was achieved by using standard linear programming library: RCaN uses lp_solve through the R package lpSolveAPI  [13] ). The third challenge was also to translate the content of a RCaN-file into matrices, and especially to allow user to specify constraints in a simple language. This was made possible by using the R package symengine  [14] , an interface to the symengine symbolic manipulation library (the syntax of explicit constraints is presented in Appendix A. RCaN can performs uniform sampling with two methods, the hit-and-run and the Gibbs sampler.
RCaNconstructor The main goal of RCaNconstructor is to help users to construct and revise CaN models in a participatory context. Internally, it ensures the consistency between the different worksheets of the RCaN-file (changing the characteristics of an object somewhere triggers the corresponding changes elsewhere). In addition, RCaNconstructor facilitates the elicitation and tracking of meta-information. RCaNconstructor can also execute RCaN commands. RCaNconstructor is a Java program using Javafx for building the graphical interface and allowing the code to be imported and run on external computers; it uses the Java library RCaller for interfacing R and Java.
Polytope building Once the components, input parameters, fluxes, observations and constraints are provided, the model is fully specified. This information is gathered in the corresponding RCaN file. The polytope can be constructed by running R commands of the RCaN library. This can be achieved within the RCaNconstructor environment or directly within the R environment. Both ways have their advantages and limitations. Using RcaNconstructor is intuitive and does not require prior knowledge of the R language. Using R provides a greater flexibility for users who are familiar with the R language. We provide below the commands in the R language. We build the polytope with following R command : It is possible to check the status of the polytope (bounded, empty or unbounded) and to compute the bounds of the polytope in every of its 775 dimensions using the following commands (the time taken to obtain the results is indicated in round brackets): 1 CanSample <-sampleCaN ( CaNMod , N =100 , thin =100 , nchain =2 , ncore =2 , method = " gibbs " ) \\ (520 s ) This command returns an R list object which contains the original model structure and two chains of 100 samples (i.
Despite the wealth of existing climate forecast data, only a small part is effectively exploited for sectoral applications. A major cause of this is the lack of integrated tools that allow the translation of data into useful and skilful climate information. This barrier is addressed through the development of an R package. CSTools is an easy-to-use toolbox designed and built to assess and improve the quality of climate forecasts for seasonal to multi-annual scales. The package 20 contains process-based state-of-the-art methods for forecast calibration, bias correction, statistical and stochastic downscaling, optimal forecast combination and multivariate verification, as well as basic and advanced tools to obtain tailored products. Due to the design of the toolbox in individual functions, the users can develop their own post-processing chain of functions as shown in the use cases presented in this manuscript: the analysis of an extreme wind speed event, the generation of seasonal forecasts of snow depth based on the SNOWPACK model and the post-processing of data to be used 25 as input for the SCHEME hydrological model.
g. : high-resolution data obtained by applying downscaling methods). These users can handle the tool by themselves, understanding each of the methodologies given the provided documentation and with the support of scientific 60 research publications. The tool is fully transparent since it is open-source, allowing the user to control, understand and even adapt every step of the analysis in depth. While simple examples are given in the package documentation, this manuscript aims to showcase the usefulness of CSTools in the context of advanced state-of-the-art use cases.
From climate data to climate information There are different forecast post-processing steps necessary to translate climate data into climate information. These steps 65 will vary depending on the applications, but usually fall within the following categories (as illustrated on Fig. 1 ): • Data retrieval and formatting: Optimal methods for spatial and temporal data manipulation, such as interpolation methods, are needed given the wide range of climate data formats. This can be a labour intensive step when trying to combine multiple datasets such as observations and forecasts from multiple systems. • Correction methods for forecast calibration: Calibration is necessary to correct systematic errors, uncover any 70 predictive signal and adjust forecasts to the observational statistical properties in order to be integrated into impact models.
The need for climate information Large multi-model seasonal forecasting systems have been developed in recent years, both from current international research projects and operational programmes. These include, for instance, EUROSIP  (Vitart et al. , 2007; Mishra et al. , 30 2019) , APEC  (Wang et al. , 2009; Min et al. , 2014) , North-American Multi-Model Ensembles  (Kirtman et al. , 2014) , and the Decadal Climate Prediction Project (CMIP6-DCPP,  Boer et al, 2016). More recently, the Copernicus Climate Change Service (C3S) funded by the European Union established a Climate Data Store (CDS) with the aim of being an authoritative source for distributing relevant datasets for a wide range of applications, including an ensemble of (mostly) European Seasonal Forecasting Systems.
, 2016a). The R packages s2dverification  (Manubens et al. , 2018) , SpecsVerification  (Siegert, 2017)  and easyVerification  (MeteoSwiss, 2017)  or the python package climpred  (Brady and Spring, 2021)  focus on skill assessment of ensemble forecasts. Climate4R  (Iturbide et al. , 2019)  is an R-bsed framework for climate data post-processing including different 100 methods. The main purpose of these different packages is the facilitation of research. CSTools, on the other hand, targets scientists interested in providing a climate product to some final users. CSTools could nonetheless be useful to research scientists, as it is made compatible some of the aforementioned R packages. 90 For a detailed description of CSTools functions and parameters, the reference manual is attached to the package and 105 available at https://CRAN.
R-project. org/package=CSTools in the standardized format of an R package documentation. In this manuscript, an overview of the methods and documentation gathered in CSTools is presented in Sect. 2, while the creation of a tailored dataset is shown in Sect. 3. Three case studies based on the analysis of an extreme wind speed event, the snow model SNOWPACK  (Lehning et al. , 2002a,b  https://models. slf. ch/p/snowpack/) and data preparation for the hydrological model SCHEME  (Baguis et al. 2010)  show the usefulness of the toolbox. Section 4 concludes this paper and 110 discusses some future developments for the package. https://doi. org/10. 5194/gmd-2021-368 Preprint. Discussion started: 6 December 2021 c Author(s) 2021. CC BY 4. 0 License.
CSTools: overview CSTools was created as part of a collaborative effort between six European institutions. Given the total number of contributors and collaborators (31 in version 4. 0), compiling all methods into a software package using the R statistical programming language (R Core Team, 2017) was considered the most suitable and versatile option. Creating an R package 115 allows the inclusion of multiple tools ranging from complex statistical and climatological methods to visualization tools in the same framework. Moreover, CSTools is open-source, thus allowing users and developers alike to benefit from lower costs and software flexibility, quality and reliability  (Information Resources Management Association, 2013). At the same time, CSTools can be integrated into other softwares in order to take advantage of its functionalities, as does, for instance, the S2S4E Decision Support Tool (https://s2s4e-dst.
125 Most functionalities exposed to the users can be invoked and applied to complex user datasets with a single function call. For example, in order to apply a given functionality named "Func", the user would write: CST_Func(dataset,. ) The "CST_*" family of functions ingest and return objects of type "s2dv_cube" (see details in Sect. 2. 1), thus allowing 130 compatibility between each functions and long post-processing chains to be created.
Technical aspects of CSTools The CSTools development guidelines have been designed to maximise compatibility with other libraries such as s2dverification, s2dv and startR, all of them designed to operate fundamentally with multi-dimensional arrays with named dimensions. Because of this design, the CSTools user is able to perform basic array inquiry on the "data" element of the 135 "s2dv_cube" objects at any point in the workflow in order to check the dimensions of the data or to find the number of members, start dates or forecast lead times analysed. Internally, each of these high-level "CST_*" functions perform two nested calls to two other different but closely related functions in the package. For example, a given functionality named "Func" would involve the following function calls: At the most fundamental level of this nested call structure, there is a call to a basic function (e.
155 This nested structure has a number of benefits: • The CSTools user code, using top-level functions, is modular, concise, readable and easy to maintain. • The R community can easily employ it via the array-compatible low-level functions. • Multi-core parallelism is straightforward to exploit via middle-level functions and high-level "CST_*" functions. • In cases where the data to process is larger than the RAM memory in the workstation or the computation is very 160 expensive, the low-level functions can be used together with the startR package  (BSC-CNS and Manubens, 2020)  to leverage HPC platforms and distribute workload in small chunks. The modular aspect of the "CST_*" functions makes it straightforward for users to create their own post-processing workflows, as shown in Sect. 3. Metadata is propagated and expanded all along the workflows.
Methods in CSTools Given that the methods included in CSTools are split into functions, the users can concatenate them to define their own postprocessing workflow. This design provides flexibility allowing the users to assess the impact of the various post-processing steps by modifying the chain of functions. The users can also select a single function and apply it outside of the CSTools workflow. The functions included in the package cover fundamental loading and transformation requirements, downscaling 170 tools, methods for correcting and evaluating forecast and advanced visualization tools (see Table  1 ). All functions are documented in a standard reference manual on the CRAN website (https://CRAN. R-project. org/package=CSTools). The documentation also includes vignettes, which are self-contained pieces of documentation combining code, text and images, describing some of the methodologies included in CSTools, as well as information on how to use the package to conduct specific analysis.