dataesr/scientific-paragraphs-categorization huggingface dataset:
Domain Adaptation In ML, learning a classifier consists on estimating h : X → Y, among a set of functions H, where X (e. g. , R d ) is the feature space and Y (e. g. , {1, • • • , n c }) is the label space. This estimation is done via risk minimization  [36] , h = argmin h∈H R Q (h) = E x∼Q [L(h(x), h 0 (x))], (1) for a loss function L, a distribution Q, and a ground-truth labeling function h 0. R Q is known as true risk. Since Q and h 0 are seldom known a priori, it is unfeasible to directly minimize equation 1. In practice, one acquires a dataset, {(x (Q) i , y (Q) i )} n i=1 , with x (Q) i i. i. d. ∼ Q and y (Q) i = h 0 (x (Q) i ) and minimizes the empirical risk, ĥ = argmin h∈H RQ (h) = 1 n n i=1 L(h(x (Q) i ), y (Q) i ).
E. Montesuma et al. / Multi-Source Domain Adaptation Through Dataset Dictionary Learning in Wasserstein Space where θ is found at each iteration via line search. In the context of MSDA,  [17]  previously defined a barycenter of labeled distributions by penalizing transport plans π (k,it) that mix classes. Mini-batch OT. For large scale datasets, computing OT is likely unfeasible, due its cubic complexity. A workaround, coming from ML, consists on using mini-batches  [10]. For M batches of size n b , this approach decreases the time complexity to O(Mn 3 b log n b ). Remark on Notation. While W 2 ( P , Q) is defined between empirical distributions, in practice it is a function of (X (P ) , X (Q) ). With an abuse of notation, the mini-batch Wasserstein distance between given random samples of size n b from P and Q is still noted as W 2 ( P , Q), with the support matrices restricted to a mini-batch.
Wasserstein Barycenters of Labeled Distributions We propose a novel algorithm for calculating differentiable Wasserstein barycenters of labeled empirical distributions. This algorithm is at the core of DaDiL (section 4), since we later represent datasets as barycenters of learned atoms. In OT, there are at least 2 ways of integrating labels, either by penalizing OT plans that transport mass between different classes  [6, 17] , or by defining a metric in the label space  [3]. We choose to integrate labels in the ground-cost, C i,j = x (P ) i -x (Q) j 2 2 + β y (P ) i -y (Q) j 2 2 , ( 9 ) where y denotes labels one-hot encoding, and β > 0 controls the importance of label discrepancy. While simple, this choice allows us to motivate the barycentric projection of  [6] , and the label propagation of  [26]  as first-order optimality conditions of W c ( P , Q), x(P ) i = T π (x (P ) i ) = n P n Q j=1 π i,j x (Q) j , ŷ(P ) i = T π (y (P ) i ) = n P n Q j=1 π i,j y (Q) j.
Dataset Dictionary Learning for MSDA In this section, we introduce our novel framework, called DaDiL, and explore how to use it for MSDA. Let Q = { QS } N S =1 ∪ { QT } correspond to N S labeled sources and an unlabeled target. Let A = [α 1 , • • • , α N S , α N S +1 ], and P = { Pk } K k=1. The Pk 's are an empirical approximation of the point clouds that interpolate distributional shift. Following our notation, α T := α N S +1. For N = N S + 1, DaDiL consists on minimizing, (P , A ) = argmin P,A∈(Δ K ) N 1 N N =1 L( Q , B(α ; P)), ( 11 ) where L is a loss between distributions. Since the target domain is not labeled, we define, L( Q , B ) = W c ( Q , B ), if Q is labeled, W2( Q , B ), otherwise, i. e. , when no labels in Q are available, we minimize the standard 2-Wasserstein distance. Optimizing 11 over entire datasets might be intractable due the complexity of OT.
19: end for 20: end for Ensure: Dictionary P and weights A. Intuition. We learn how to express each distribution Q ∈ Q as a barycenter of free distributions P = { Pk } K k=1 , parametrized by their support i. e. , (X (P k ) , Y (P k ) ). In other words, we learn P s. t. Q is contained the Wasserstein hull of atoms, M(P). Implementation. We implement algorithms 1 and 2 using Pytorch  [22]  and Python Optimal Transport (POT)  [12] , for automatic differentiation and OT details respectively. As previous works  [17, 35] , DaDiL is applied to the latent space of an encoder, pre-trained on source domain data, as shown in figure  1. Computational Complexity. In algorithm 2, the complexity of line 13 dominates over other lines. As we discussed in section 4. 1, the complexity of calculating B(α ; P) depends on the size of distributions support.
Multi-Source Domain Adaptation Experimental Setup. All experiments were run on a computer with a Ubuntu 22. 04 OS, a 12th Gen Intel(R) Core TM i9-12900H CPU with 64 GB of RAM, and with a NVIDIA RTX A100 GPU with 4GB of VRAM. We explore the following hyper-parameters, The complexity of our model is controlled through n and K. We provide further analysis on the robustness w. r. t. hyper-parameter choice, as well as the full set of chosen hyper-parameters in our appendix. • For other algorithms from the State-of-the-Art (SOTA), we use the best hyper-parameter settings reported by their respective authors. Caltech-Office 10 is a benchmark consisting on the intersection of the Caltech 256 dataset of  [14]  and the Office 31 dataset of  [30]. It has 4 domains: Amazon (A), dSLR (D), Webcam (W) and Caltech (C).
On the other hand, the linear average of distributions can be understood as importance weighting on samples. Under the covariate shift hypothesis, re-weighting samples is enough, but under more complicated shifts (i. e. non-linear data transformations), Wasserstein barycenters are more flexible. Shallow vs. Deep Domain Adaptation. As remarked by  [35] , the assumption of having a meaningful feature extractor φ before performing DA is realistic, as in modern practice pre-trained models are widely available. It is noteworthy that a fine-tuning step with source-domain data may be necessary in order to achieve better performance. In addition, doing so allows for the comparison with deep DA methods. In this context, we remark that our method improves over previous deep DA SOTA in the context of the Refurbished Office 31 and CWRU benchmarks.
15%, 2. 29% and 7. 71% respectively. Moreover, we show that general interpolations in the Wasserstein hull of our learned dictionary can be equally interesting for MSDA. Our framework opens an interesting line of research, for learning empirical distributions, generating synthetic through Wasserstein barycenters and interpolating distributional shift in Wasserstein space. It is flexible so as to accommodate other notions of barycenters of distributions, and loss functions between reconstructions and real datasets. In practical terms, future works will focus on parametric formulations of DaDiL. In theoretical terms, we seek to understand the statistical challenges posed by DaDiL.
The reactive transport code CrunchClay was used to derive effective diffusion coefficients (D e ), clay porosities (ε), and adsorption distribution coefficients (K D ) from through-diffusion data while considering accurately the influence of unavoidable experimental biases on the estimation of these diffusion parameters. These effects include the presence of filters holding the solid sample in place, the variations in concentration gradients across the diffusion cell due to sampling events, the impact of tubing/dead volumes on the estimation of diffusive fluxes and sample porosity, and the effects of O-ring-filter setups on the delivery of solutions to the clay packing. Doing so, the direct modeling of the measurements of (radio)tracer concentrations in reservoirs is more accurate than that of data converted directly into diffusive fluxes.
With the present contribution, we would like to refute assumptions (3) to (  5 ) that we consider are incorrect and overcome assumptions (1) and (  2 ) by providing the community with a user-friendly, fast, generic, reliable, and free tool, namely the CrunchEase user interface to be used with the CrunchClay code for use interpreting diffusion data with D e -ε-K D conceptual models. We will also show that CrunchEase in conjunction with CrunchClay can be used as a first modeling step that can help to set up a more complex modeling approach that takes into account state-of-the-art theoretical concepts for diffusion and surface processes of contaminants in clayey materials.
Reactive transport model and investigated systems Reactive transport simulator and user interface. Reactive transport calculations of this study were carried out with the code CrunchClay. CrunchClay code capabilities, equations, and the solver scheme have been described in detail in previous publications  15, [17] [18] [19] [20]. A graphical user interface, CrunchEase, has been designed to help users build CrunchClay input files and to extract radionuclide (or another tracer) diffusion results. This further allows the readers to re-run the simulations shown in the present study, and to easily use the code to fit their experimental diffusion data. The CrunchEase package is freely available. Installation instructions and snapshots describing the interface are available in the supplementary information.
Conclusions We have demonstrated that a reactive transport modeling approach that includes the full geometrical and operational complexity of an experiment, such as heterogeneities and time-dependent sampling effects is a very powerful tool for the interpretation of diffusion data. While a classical D e -ε-K D model may be implemented    S1  in the supplementary materials. in any reactive transport code, the development of the new graphical user interface CrunchEase makes it easy to apply by experimentalists without background in reactive transport modeling. CrunchEase can also be used to fit through-diffusion data from experiments with changing boundary conditions (no reservoir replacement; not shown in this paper). In the future, the code may be easily extended to other types of diffusion experiments, such as in-diffusion experiments.
Since experimental biases can be modeled explicitly, the accuracy of calculated diffusion parameters with a reactive transport approach is expected to be better than for the application of analytical solutions, which cannot include the full complexities of real experiments. The numerical efficiency of CrunchClay makes our approach suitable to fit model parameters very quickly. For instance, for 1D problems simulating a diffusion experiment with an experimental time frame of approximately 50 days, the model running time is most often less than 1 min. This approach could also be used easily with parameter estimation softwares such as PEST  32. On a final note, CrunchEase makes it easier to transition from a D e -ε-K D modeling approach to a reactive transport modeling approach taking into account surface-enhanced diffusion processes and advanced chemical reactivity models such as surface complexation models.
As data is a predominant resource in applications, Riemannian geometry is a natural framework to model and unify complex nonlinear sources of data. However, the development of computational tools from the basic theory of Riemannian geometry is laborious. The work presented here forms one of the main contributions to the open-source project geomstats, that consists of a Python package providing efficient implementations of the concepts of Riemannian geometry and geometric statistics, both for mathematicians and for applied scientists for whom most of the difficulties are hidden under high-level functions. The goal of this monograph is two-fold. First, we aim at giving a self-contained exposition of the basic concepts of Riemannian geometry, providing illustrations and examples at each step and adopting a computational point of view.
This monograph aims at providing the computational tools to perform statistics and machine learning on Riemannian manifolds to the wider data science community. The work presented here forms one of the main contributions to the open-source project geomstats, that consists in a Python package providing efficient implementations of the concepts of Riemannian geometry and geometric statistics, both for mathematicians and for applied scientists for whom most of the difficulties are hidden under high-level functions. Other Python packages do exist and mainly fall under one of two following categories. On the one hand, there are the packages that focus on a single application, for instance on optimization: Pymanopt  (Townsend et al. , 2016) , Geoopt  (Becigneul and Ganea, 2019;  2 Differentiable manifolds.
Implementation in geomstats Now that the fundamental notion of manifold has been exposed, we can delve more into the architecture of the geomstats package, and summarize the choices that we made in its development. Firstly the package is organized in different modules that distinguish between the geometric and the statistical operations. There is thus a geometry module, that gathers all the implementations of manifolds, connections and Riemannian metrics, and a learning module where estimation algorithms are implemented in a generic fashion and take the geometric structure and the data as inputs. The goal is that all the learning algorithms can be run seamlessly on different manifolds, and with different metrics. Basic sampling schemes are available, and a more extended sampling module is currently being developed with the same spirit.
A visualization module allows to plot data on the common manifolds in dimension two or three. In this section, we focus on the geometry module, and more specifically on the objects that represent manifolds. The Riemannian metric objects will be described in Section 3. 1 along with the mathematical definitions. The statistical and learning tools will be described in Section 6. The package is object-oriented in the sense that all the tools are implemented as classes that contain all the methods related to a tool. Object-Oriented programming (OOP) is a programming paradigm that consists in grouping properties and functions related to a common concept into an object, called a class in Python  5 The geometry module of geomstats is organized by classes that each represent a geometric structure. To guarantee the consistency of all the classes, we implement an abstract parent Manifold class, and the actual implementations of the usual manifolds are subclasses of this parent class.
The aim of abstract base classes is to provide the minimal skeleton of attributes and methods expected in its subclasses. The methods of the abstract class are thus declared but contain no implementation, and they are overridden by the subclasses. The manifold classes A subclass of Manifold gathers the methods to work with data lying on the considered manifold. Note that these classes do not explicitly provide a representation of the manifold (as e. g. a triangulated surface in 2d), but the tools to handle points and tangent vectors. As we work with embedded manifolds in most cases, points and tangent vectors are themselves represented by multi-dimensional arrays. These are either NumPy arrays, or TensorFlow of PyTorch tensors according to the backend that is being used. Mathematically, the first attribute of a manifold is of course its dimension, called dim for brevity throughout the package.
Application to Kendall shape spaces The following subsection is inspired by  Nava-Yazdani et al. (2020)  but constitutes an original contribution by putting all the code open source in our package geomstats, and by improving the implementation of parallel transport. It is a collaboration with Elodie Maignant and was presented at GSI 2021 in  Guigui et al. (2021). We revisit Example 2. 5 (page 7) with the additional idea that the size of a set of landmarks may be filtered out to define a shape. The study of these spaces, including their mathematical structure, the properties of statistical distributions and estimation methods for shape data and their applications to many scientific fields goes back to the late 1970's with the works of Mardia, Bookstein and Kendall among others. For historical notes on this research area and an introduction to the applications Dryden and Mardia (see 2016, Preface and Section 1.
This formulation allowed to easily implement parallel transport on Kendall shape spaces, and is to the best of our knowledge the only open-source Python implementation of these spaces. The space of correlation matrices seen as the quotient of SPD manifolds by the action of diagonal matrices is also implemented in geomstats, and others spaces such as the space of positive semi-definite matrices will be implemented in the near future. Two particular instances of quotient spaces were then introduced: reductive homogeneous spaces and symmetric spaces. They are in fact direct generalizations of Lie groups and many operations from quotient spaces simplify. These concepts are however not used in the present implementation, as their properties actually allows to derive more efficient closed-form solutions.
Statistics and machine learning with Geomstats To conclude this section, we demonstrate the use of geomstats to perform statistics on manifold-valued data. The strength of the package is that learning algorithms are defined as external estimators that take the geometric object as input. This happens thanks to the standardized interface of the classes, and ensures that all learning tools be available for all the manifolds and metrics. The obtained flexibility allows to compare the impact of the different metrics on the learning results. We give a brief introduction to geometric statistics, and the interested reader is referred to  Pennec et al. (2020, Chapter 2)  for the theoretical exposition. An introductory paper gathering more examples using geomstats was presented at the Scipy Conference 2020 in Austin, Texas  (Miolane et al.
If the manifold is embedded in some R N and equipped with the pullback metric, it can be obtained by projecting the usual gradient to the tangent space at the point where the function is being differentiated, similarly to the Levi-Civita connection. By definition, the Fréchet mean must be a critical point of the sum-of-squared distances function, so that it verifies A simple strategy to estimate the sample Fréchet mean is thus to use a fixed-point iterative algorithm until Equation (6. 1) is verified. An update at iteration t is performed along a geodesic: x t+1 = Exp x t (γ n i=1 Log x t (y)), where γ is a step size. Higher-order methods use adaptive step size, approximates of the Hessian function or its link with curvature to improve the performance of the estimation. Some of these are implemented in the FrechetMean module of geomstats.
One can also generalize the K-means clustering algorithm and the minimum distance to mean (MDM) classification algorithm with Fréchet means. These are implemented in the classes RiemannianKMeans and RiemannianMinimumDistanceToMeanClassifier in geomstats. Moreover, one can use the Fréchet mean to linearize the data by lifting it to the tangent space at the Fréchet mean, i. e. to consider the transformed data set (x 1 ,. , xn ) where xi = Log x(x i ). This is equivalent to using a Taylor expansion at the first order of the Riemannian exponential map around 0: x i = Exp x(x i ) = x + xi + O(t 2 ). As T xM is a vector space, all the usual statistical and machine learning tools can be used off-the-shelf on the transformed data set. This is implemented as a Transformer from the scikit-learn package in the ToTangent class of geomstats, and can be used in scikit-learn's pipeline.
, 2010). The projection of a data point x on a geodesic γ with initial velocity v at the mean x is defined by There is no guarantee that this projection exists, but if the data is not too spread out, one can hope that there exists a convex neighborhood where the exponential map is injective that contains x and a portion of the geodesic. The projection is solved by a gradient descent, as in the case of the Fréchet mean, where the gradient of the exponential is computed by automatic differentiation. The next step is to minimize the overall reconstruction error, given a dataset x 1 ,. , x n , we look for the initial velocity of a geodesic as:. 3) This time computing the gradient of the objective function seems more complicated, as n minimization problems are already solved by a gradient descent to evaluate the function.
We exemplified the geometric concepts with the most common manifolds encountered in mathematical text books as well as in applications. Our examples include code snippets using geomstats to demonstrate how to leverage differential geometry in practical use cases. Lastly, we gave an introduction geometric statistic tools such as the Fréchet mean, principal geodesic analysis and geodesic regression. We illustrated these on toy examples using synthetic datasets on manifolds. These geometric statistical tools are implemented in the geomstats package with a common high-level interface, following the Scikit-Learn syntax. Consequently, geometric statistics become available to any data scientist on a wide variety of practical problems. We hope that the concepts presented here will drive scientists to use, and contribute to, geometric statistics with the geomstats library.
Future developments of the geomstats package will integrate additional statistical methodologies published by the geometric statistics community, such as wrapped Gaussian processes, multivariate and polynomial geodesic regression. We will also extend the scope of geomstats and include computational methods for information geometry, a field at the intersection of geometry and statistics closely related to geometric statistics. Another module on stratified spaces such as graph and tree spaces is currently being developed, and raises fundamental methodological questions as these spaces are not smooth manifolds but unions of smooth manifolds equipped with a distance function. Together, these advances aim to provide mathematically-grounded foundations for computational geometric statistics.
Acknowledgment This work was funded by the  ERC  grant Nr. 786854  G-Statistics  from the  European Research Council  under the European Union's  Horizon 2020 research and innovation program. It was also supported by the  French government  through the  3IA Côte d'Azur Investments   ANR-19-P3IA-0002  managed by the  National Research Agency. The authors are grateful to  Adele Myers  for her corrections and suggestions to improve this monograph.
We here present a database of evidence on the impact of agricultural management practices on biodiversity and yield. This database is the result of a systematic literature review, that aimed to identify meta-analyses that use as their response variables any measure of biodiversity and yield. After screening more than 1,086 titles and abstracts, we identified 33 relevant meta-analyses, from which we extracted the overall estimates, the subgroup estimates as well as all information related to them (effect size metric, taxonomic group, crop type etc. ). We also extracted information relative to the empirical studies used for each meta-analysis and recorded the countries in which they took place and assessed the quality of each meta-analysis. Our dataset is publicly accessible and can be used for conducting second-order metaanalyses on the effect of management measures on species richness, taxon abundance, biomass and yields.
In addition, we describe the procedure we followed to compile this dataset and the R scripts used for the tables and figures. By exploring and analyzing these datasets, users can identify knowledge gaps and create evidence maps. • Scientists in the fields of agronomy, landscape ecology and other related fields can reuse and/or combine our datasets with other data, in order to draw general conclusions about the effects of management practices on both biodiversity and yield. In addition, practitioners and decision-makers can use our data too, in order to obtain a comprehensive overview of current evidence. • The data can be used to identify the management practices that ensure win-win scenarios in agricultural management. Indicatively, the data can be used in a mixed-effects model that will unveil the relative importance of variables such as crop type, functional group etc.