dataesr/scientific-paragraphs-categorization huggingface dataset:
Data archiving server The EPICS control variable storage server for ITER has been implemented on the basis of a plugin-based architecture with the aim of improving maintainability and code reusability. In addition to data storage services from SDN (ITER CODAC Synchronous Data Network), EPICS Channel Access and EPICS pvAccess protocols, ITER CODAC includes data streaming services to disseminate the same data over other systems. Thanks to this architecture it is possible to serve different types of data without the need to maintain different software. As shown in Fig. 5 , the architecture of the storage system is a classical frontend-buffering-backend architecture consisting of a front-end subsystem that receives data from the appropriate network and protocol, a buffering and queuing subsystem that stores the received data waiting to be processed, and a back-end subsystem that processes the buffered data and processes them according to the required functionality.
This architecture implements most of the logic common to many frontend-bufferbackend systems and leaves it to the plugins to implement a small set of actions specific to the solution they support. This makes it easy to develop new services, to extend them to new data types, as well as to maintain those already implemented, since improvements and repairs are common to all of them. A fundamental component of the system is the Nested Datatype Object (NestedDT). This internal object implements a common generic datatype that allows the same architecture to be maintained for very different datatypes. For this purpose, the data, which are received by the listeners, are translated into NestedDT and from this point on can be known and processed by the whole system, including the chosen backend plugin.
Backend and archiving plugins The backend subsystem operates on a control loop in which it is able to detect new data in its buffers and pass it to the active plugin for proper processing. In addition, it implements timeout detection actions to force data flushing in case it is necessary, as there are storage plugins that perform some internal buffering in order to optimize write actions. Also, the backend implements asynchronous file rotation mechanisms that avoid backend locking. In the case of the Channel Access PVs storage plugin, the rotation mechanism has to create a new file which (as previously mentioned) in the case of ITER can exceed 50 K variables, and this action requires a non-negligible amount of time. For the implementation of the non-blocking rotation, and as shown in Fig. 7 , we have chosen to incorporate an asynchronous procedure carried out by a thread created ah-hoc.
Conclusions A storage service for ITER EPICS control variables (process variables or PVs) has been designed and implemented. This service is not only able to efficiently store the variables published through the classical "Channel Access" protocol, but also allows the storage of the EPICS pvAccess control variables (incorporated from version EPICS version 7) whose complexity lies in a highly nested data type that can also vary dynamically over time. The presented solution uses HDF5 files (which are de facto standard in ITER) and implements a data model that, on the one hand, preserves the original structure of the variable and, on the other hand, optimises the reading of certain fields of a variable, as it avoids the complete reading of the whole structure. In addition, this storage service is based on a frontend-buffering-backend architecture with plugins that makes it easy to develop new services by simply implementing the corresponding plugins.
• File: Main component that groups and contains the rest of elements of the model. • Group: a collection of objects (or other groups). • Dataset: a multidimensional array of data elements with attributes and other metadata. The dataset component is crucial in HDF5 library because the data read and write operations are optimized for every dataset and are decoupled of the rest of datasets. • Datatype: a description of a specific class of data element. The most significant families or classes of HDF5 datatypes are: integer, floating-point, string, bitfield, opaque, compound, enum and array. HDF5 allows to define new datatypes by combining other types.
They can also identify pre-eruption time-series patterns in large datasets  27  that may otherwise be unclear or not appropriately emphasized by human operators. ML can also help to mitigate human biases when weighing the statistical evidence of a precursor  17  , although it risks introducing its own biases, e. g. , during sampling, labeling, data, and algorithms selection  28, 29. ML can be used to analyze various types of data, including seismic  30, 31  , gas  32  , and satellite data  33  , which supports an increasingly comprehensive characterization of volcanic activity.
Using data from multiple volcanoes for generalized forecasting Our study identifies sets of precursor signals of eruptions that are shared amongst groups of volcanoes. Doing so supports the development of global forecasts that can be applied at analogue volcanoes with no historic eruption data. To achieve this, we developed a new type of machine learning pipeline that shares data from many volcanoes to identify common seismic patterns that recur preferentially before eruptions at different volcanoes. We tested the forecasting ability of these precursors by training global, multiple volcano forecast models and seeing how they performed compared to models for individual volcanoes. In particular, our machine learning pipeline consists of a random forest classifiers whose individual decision trees were trained on diverse, but balanced, subsets of the statistically significant precursors, which reduces overfitting to specific volcanoes or eruptions.
The resulting model, when fed new seismic data, outputs a forecast number between 0 and 1 with higher numbers indicating a higher likelihood of eruption  34. We used a dataset comprising 41 eruptions across 24 volcanoes (Fig  1a ) with a combined seismic record length of ~73 years. Based on eruption type, we defined three pools of volcanoesmagmatic (9 volcanoes, 16 eruptions), phreatic (6 volcanoes, 15 eruptions), and global (all volcanoes and eruptions)and tested the similarity of their pre-eruption signals (Fig 1b; see Methods). We were interested mainly in short-term changes that are useful for responsive monitoring and to inform evacuation orders. Therefore, we focused on signals that occurred 48-hours prior to eruptions using a technique called time-series feature engineering  27, 17  , which applies brute-force pattern matching to the windowed data.
Both strategies are important to assess and understand how generalized forecasters would perform at wellmonitored volcanoes with eruptive records (CVE) and those with no historic monitoring (CVV). We also compared generalized models to (1) tailored models trained exclusively on the data from the target volcano (for volcanoes with three or more recorded eruptions), and (2) an optimized seismic-amplitude (RSAM) model. Forecast performance is assessed in terms of the reliability and discriminability of the models on test datasets. Reliability refers to a model's capacity to anticipate eruptions and, hence minimizing missed eruptions. Discriminability measures the model's ability to distinguish between eruptive and non-eruptive periods, hence minimizing false positives. Both aspects need to be considered when developing and implementing forecasters across many types of natural hazards.
Improving decision support for eruption warning systems The purpose of eruption forecasts is to provide decision-makers objective, preferably quantified information about the likelihood of an eruption in a specified warning period  39, 5. Here, we consider that warnings could be triggered when the model output exceeds a threshold and end once it has dropped below the threshold for more than 48-hours. As an example, applying the phreatic pool model to the 11-year dataset at Whakaari, for a threshold of 0. 8, results in 80% of eruptions (4 out of 5) occurring inside a warning period and a total warning period of 11% (i. e. , 441 days). This is approximately a 2% chance of eruption in a 48-hour period when a warning is in effect and a 0. 06% chance outside the warning period (see Methods). These conditional probabilities have low absolute values and thus may be better expressed as hazard ratios, which quantify forecast skill in terms of the relative likelihood of an adverse event under different warning conditions.
We conclude that generalized or transferable forecast models that exploit ergodic features in the underlying data are ready to contribute to volcanic forecasting. In addition, the global move towards open hazard monitoring data will allow much easier aggregating of data from multiple regions to into comprehensive training datasets  49, 50  that will improve the accuracy and usefulness of forecasts across multiple hazards. 2 Yamaoka, K. , Geshi, N. , Hashimoto, T. , Ingebritsen, S. E. & Oikawa, T. Special issue "The phreatic eruption of Mt. Ontake volcano in 2014". Earth, Planets Space 68, 1-8 (2016).
ACKNOWLEDGEMENTS We acknowledge GEONET (New Zealand) and AVO ( Alaskan Volcano Observatory, USA ) seismic monitoring system for supplying free access to their volcanic seismic data. We wish to thank  NZ MBIE   UOAX1913  ( Transitioning Taranaki  to a Volcanic Future) for support of AA, DD, and SJC during this study. We wish to thank  NZ MBIE   E7774  ( Adapting  to climate change through stronger geothermal enterprises') for support of AA, DD, and SJC during this study. LC thanks the support of the ' Programa de Riesgo Sísmico' (PRS, Actividades de Interés Nacional, Universidad de Chile). SP was supported by the  New Zealand Ministry of Business, Innovation & Employment (MBIE)  through the  GNS Science Hazard & Risk Management programme (Strategic Science Investment Fund , contract  C05X1702 ).
DATA AND CODE AVILABILITY The codes required to replicate the results of this study are available at https://github. com/ddempsey/puia, released under the MIT license. Codes are provided in the Supplemental Material to compute the ROC curves for all three types of foresters. Raw waveform data for New Zealand volcanoes can be downloaded from GEONET (https://service. geonet. org. nz); and for Alaskan volcanos from IRIS (http://service. iris. edu). Both are operable as clients though the FDSN webservice (https://www. fdsn. org/networks/). The seismic data streams used in this study (rsam, mf, hf, dsar) for the New Zealand and Alaskan volcanoes are available as supplementary material in the form of. csv files. If you utilize the provided seismic data, please cite this article accordingly.
Machine learning pipeline (workflow) We used continuous seismic data from 24 volcanoes, with one station per volcano (as shown in Fig  1a  and listed in Supplement Table  S2 ). Where multiple stations were available, we selected the station that had recorded the most eruptions and was closest to the eruptive vent. The dataset includes more than 40 major eruptive episodes. The majority have been classified by the Global Volcanism Program, 2023. We excluded minor eruptive activity, such as geysering and passive ash emissions, as our focus was on the initial abrupt, impulsive (phreatic and magmatic) onset of eruption. The machine learning pipeline  27, 34  is summarized below. Data preprocessing. We use the vertical velocity component time series at each station with the instrument response removed.
We compute Median Frequency (MF) and High Frequency (HF) data streams in the same way as RSAM but instead filtering in the range of 4. 5 to 8 Hz and 8 to 16 Hz, respectively. These two data streams focus on signal attenuation effects above the frequency range tremor energy is commonly radiated  16. Finally, Displacement Seismic Amplitude Ratio (DSAR) is calculated as the ratio of the integrals of the MF and HF signals  17. High values of DSAR have been inferred to correlate with high gas levels in the edifice, suggesting either reduced fluid motion and/or trapping that has led to a gas-accumulation  17, 51   (Ardid et al. ,     2022-23). Time series gaps were imputed by linear interpolation and because these gaps occur in noneruptive periods, a later step of down sampling reduces their impact on the models  27.
Labelling. We define a classification problem that prioritizes the algorithm attention on signals occurring in the 48-hours before eruptions, called the "look forward" window. All feature windows whose end time occurs 48 hours or less before the date of an eruption onset are labeled '1', while the remaining windows are labeled as '0'. For training windows that overlap 75%, this results in 160 label-1 windows (pre-eruptive; four for each of the 40 eruptions in the dataset) and more than 50 000 label-0 windows (non-eruptive). The large imbalance is handled by randomly discarding non-eruptive '0' windows until the dataset is balanced. This is repeated many times to reduce the effects of randomness on downsampling. Feature ranking. For a given down-sampled window subset, feature values are subdivided into groups corresponding to label-0 and label-1.
Pseudoprospective forecast performance is not guaranteed to replicate real-time prospective performance. In machine learning, cross-validation is used to evaluate model performance as well as its ability to generalize to new data or contexts. Leave-One-Out Cross-Validation (LOOCV) trains a model using all samples (volcanoes or eruptions) in a dataset except for one, and then using the excluded sample as a validation set to measure model performance. This is repeated for each sample in the dataset and the performance of all models is combined for an overall estimate of model performance. Here, we applied LOOCV at both the volcano (CVO) level to construct global ML forecasts and at the individual eruption (CVE) level for the comparison to tailored models (Fig  1c ). For the latter, a single out-of-sample forecast at a volcano is assembled from multiple CVE forecasts of its individual eruptions.
It permitted their effects, such as a spurious strong infall onto the reconstructed Local Volume, to be eliminated. Later, it allowed for simulations of local clusters such as our closest neighbor, the Virgo cluster of galaxies  (Sorce et al. 2016a (Sorce et al. , 2019 (Sorce et al. , 2021)) , as well as other local clusters (e. g. , Centaurus, Coma, and Perseus, Sorce 2018;  Sorce et al. 2023). Even more recently,  Graziani et al. (2019) ;  Boruah et al. (2022)  and  Valade et al. (2022) , borrowing from  Lavaux (2016) , proposed Bayesian techniques that could reasonably reduce the infall onto the reconstructed Local Volume. As with any Bayesian techniques, they relied on theoretical expectations, in their case: those mostly coming from the ΛCDM cosmological paradigm. However, they also relied (heavily) on prior knowledge of the dataset.
From a broad point of view, it is a typical inverse problem regularly solved in various fields (including in astrophysics), for instance, for image restoration (e. g. Van Lieshout 1994; Bijaoui 2013 ). In our case, functions used within the core of the algorithm should not rely heavily on the cosmological model nor on the dataset configuration. This makes it easier to: 1) change for another cosmological model; 2) change for another dataset; and 3) disentangle the underlying signal in the data from the signal that is induced by priors. More specifically, galaxy radial peculiar velocities are derived via a cosmological model from galaxy distance moduli and observational redshifts, with the latter being far more precisely determined than the former and uncertainties on the latter usually considered negligible.
Subsequently, the minimization of both energy terms for a given realization of the dataset corresponds to a maximum of the probability density, that is, to a bias-minimized catalog: a new set of distance moduli and their uncertainties, ( μ, σ). In other words, we need to minimize with a given set of parameters: ( μ) = arg min{U D (µ|d, c) + U I (µ|d, c) -logp(c)}. ( 6 ) This can be solved sampling the Gibbs point process  (Chiu et al. 2013 ) within a simulated annealing algorithm.
For each data point in the observational catalog, we find all the halos such that |z obsz obs, datapoint | < 0. 01, then sort these points by this value and by |sglsgl datapoint | and |sgbsgb datapoint |. We take the first halo of the sorted list as the mock point for the observational data point. The obtained (sgl,sgb,z obs ,µ) set constitutes the true synthetic catalog. We add an uncertainty to the halo distance modulus as µ = µ + R σ datapoint. We assign σ datapoint as the uncertainty for the halo distance modulus, then R is drawn from a Gaussian distribution of mean zero and variance one. We note that because the observational catalogs consist in a collection of distance moduli obtained with different indicators, coupled with the fact that several distance modulus estimates may be available for a given galaxy and a fortiori for groups and clusters, the σ datapoint ensemble spreads over a large range of values rather than being unique for all the data points.
Results Figures 4-7 present the results of the above-described algorithm applied to one of the biased mock catalog mimicking cosmicflows-3. The yellow color stands for true data points, namely, without errors (true catalog), while the black color is used for the catalog with errors (biased catalog) used as input for the algorithm. The blue color is used for data points after n mh Metropolis-Hastings iterations of every single one of the n points (called iteration 1) and the red color after n mh × n sa iterations, namely, the Metropolis Hastings samplings embedded into the simulated annealing algorithm (in other words, iteration n sa , hereafter denoted as "corrected"). In the following, figures and the associated results are described more thoroughly. We note that although the ultimate goal is to obtained biasminimized peculiar velocity catalogs, since the input consists in distance moduli (that permit deriving ultimately peculiar velocities), analyses are conducted on the former as well as on the latter and on distances.
At small distances, the average error becomes more centered on zero, a clear indication that a systematic -galaxies too close on average as per bias b2has been decreased. We note that the more data points there are in a given region, the better the algorithm performs. This is in agreement with the data interaction energy term. It proves that this term is essential and that it enforces the small-scale velocity correlation in the interaction shapes. It also confirms that data points cannot be considered individually but together as a whole: the bias-minimized catalog. The right panel shows for information that newly assigned uncertainties and true errors on data A1, page 9 of 25 Fig. 5. Comparisons of spatial distribution properties in the datasets. Left: histograms of the ratio of the differences between biased (black solid line), after iteration 1 (see text for a definition, blue dotted line), corrected (red dashed line), and true distance moduli to the true distance moduli, namely, the errors in percent on distance moduli.
Middle: difference between true and biased (black diamonds), after iteration 1 (blue square) and at the end of the process or corrected (red circles) distance moduli versus true distance moduli. Right: ratio of the difference between assigned uncertainties and true errors to the true errors on distance moduli. Same colors and line styles as the left panel. Fig. 6. Comparisons of velocity properties in the datasets. Left: true (thick yellow solid line) and biased (black solid line) peculiar velocity distributions vs. those derived from after iteration 1 (blue dotted line) and at the end of the process or corrected (red dashed line) distance moduli. Middle: histograms of errors in absolute value on peculiar velocities derived from before correction, after iteration 1 and after correction distance moduli.
In addition, bias b4 is fully present and more difficult to deal with. While it is possible to prevent from having it when starting from distance moduli to derive peculiar velocities, it is more difficult to extract non-b4-biased distance moduli from peculiar velocities Fig. 7. Comparisons of dataset properties. Left: median of the difference between distances obtained from after iteration 1 (filled blue squares) and at the end of the process or corrected (filled red circles) distance moduli and those derived from biased distance moduli as a function of the true distances. Errors on the median are obtained with bootstrapping. New distances tend to be larger than initial ones. This is in agreement with the reduction of the homogeneous Malmquist bias that statistically tends to put objects closer than they are.
Observational catalog We apply the algorithm to the third dataset of the Cosmicflows project  (Tully et al. 2016 ). This catalog contains 17 649 galaxy A1, page 13 of 25 Fig. 10. Comparisons of the velocity fields reconstructed from the different datasets. Left: variances (black color) and slopes (red color) of linear fits to cell-to-cell simulated/reconstructed velocity field comparisons considering larger and larger sub-volumes of the total boxes. The variance, namely, the difference between the reconstructed and simulated velocities, is the greatest when using the biased catalog (black crosses) -especially when considering the largest sub-volumes -and the smallest when using the true catalog (black circles) to reconstruct the fields. It has intermediate values when the corrected catalogs (black stars, squares, and diamonds) are used to reconstruct the fields.
In a future paper of this series, we will study the impact of the cosmological parameter value choice in thorough details. Still, since  Tully et al. (2016)  estimate the third dataset of the Cosmicflows project to be compatible with H 0 = 75 ± 2 km s -1 Mpc -1 , Appendix A shows the result obtained with WMAP7-like parameters: H 0 = 74 km s -1 Mpc -1 , Ω m = 0. 27, and Ω Λ = 0. 73. Additionally, Sect. 6 presents a quantification of the significance of the difference between results obtained with this second choice of cosmological parameter set values versus the first one (Planck-like).
These infalls and rounded structures are the result of biases that the Wiener-filter cannot not take into account by itself. In reconstructions obtained with bias-minimized catalogs, structures are more sharply defined. The velocity field presents several islands of convergence or divergence, in agreement with the clustering of galaxies as given by the redshift survey. We note that the central outflow is more pronounced than the overall infall in the reconstruction obtained with the biased observational catalog than in that obtained with the biased synthetic catalog. This is because we use in both cases the same Planck Hubble constant value, while the cosmicflows-3 catalog zeropoint is set to a higher local Hubble constant value. The detailed effect of the Hubble constant choice and a possible estimation of the best fit to the data will be thoroughly investigated in a subsequent paper.
This concept is at the core of the algorithm proposed. Moreover, the model, tailored for this purpose, does not rely on very detailed prior knowledge of the catalog nor prior hypotheses specific to the catalog: -It offers a great flexibility, -It does not require as many priors as usual Bayesian techniques used in the field, -These priors are independent on the datasets unless the data sampling in space varies from one extreme to another, -The algorithm thus makes it possible to easily switch from one dataset to another, -The cosmological model and its parameters can also be modified in the most convenient way 7. The proposed method diminishes the effects of the biases. The obtained results and the conducted statistical tests show the reduction of the effects of the biases in two situations.