dataesr/scientific-paragraphs-categorization huggingface dataset:
Experimental Setup Experimental platform. We perform all the experiments on a cluster of 20 machines. Each machine is an Intel Xeon CPU E5520 @2. 26GHz with 32 GB memory. The machines are connected through a 2xGigabit Ethernet (Broadcom NetXtremeII BCM5716). Datasets. We now provide an overview of the datasets used in our experiments. Amazon. We use two sets of real traces from Amazon datasets  [25] : movies and books. We use the ratings for the period from 2011 till 2013. The movies dataset consists of 1,671,662 ratings from 473,764 users for 128,402 movies whereas the books dataset consists of 2,708,839 ratings from 725,846 users for 403,234 books. The ratings vary from 1 to 5 with an increment of 1. The overlapping users in these two datasets are those Amazon users who are present in both datasets and are ascertained using their Amazon customerids.
The number of such overlapping users from both the domains is 78,201. Movielens. We use the Movielens dataset (ML-20M) for evaluating performance of X-Map within a single domain. This dataset consists of 20,000,263 ratings from 138,493 users for 27,278 movies. In this dataset, the ratings also vary from 1 to 5 with an increment of 1. Evaluation metrics. We evaluate X-Map along three complementary metrics: (1) the recommendation quality as perceived by the users in terms of prediction accuracy, (2) the degree of privacy provided to the end-users in terms of the privacy parameters ( , ), and (3) the scalability in terms of speedup achieved in X-Map when increasing the number of machines in the cluster. Accuracy. We evaluate the accuracy of the predictions in terms of Mean Absolute Error (MAE).
MAE computes the average absolute deviation between a predicted rating and the user's true rating. MAE is a standard metric used to evaluate state-of-the-art recommenders  [16, 31]. We assume that the predicted rating for an item i is denoted by pi and the actual rating is denoted by ri in the test dataset. Then, the MAE for a test dataset, with N ratings, is computed as: N i=1 |p i -r i | N. Given that rmin and rmax denotes the minimum and maximum ratings respectively, the following inequality always holds: 0 < M AE < (rmax -rmin). The lower the MAE, the more accurate the predictions. Privacy. Our differential privacy guarantees are parameterized as follows: for the PRS technique (Algorithm 3) used for AlterEgo generation and for the PNCF (Algorithm 5) used for the private recommendation generation in X-Map.
Sp = T1 Tp Due to the considerable amount of computations for heterogeneous recommendation on the Amazon dataset, we compare the speedup on p processors with respect to a minimum of 5 processors (T5) instead of a sequential execution (T1). Competitors. We now present the recommenders against which we compare X-Map. Existing recommendation schemes over several domains can be classified as follows. Linked-domain personalization. The goal here is to recommend items in the target domain (D T ) by exploring rating preferences aggregated from both source and target domains, i. e, to recommend items in I T to users in U S based on the preferences of users in U S ∪ U T for items in I S ∪ I T. In this approach, ratings from multiple domains are aggregated into a single domain. Then, a traditional CF mechanism can be applied over this aggregated single domain  [11, 29].
Finally, user-based collaborative filtering is performed. Baseline prediction. For a sparse dataset, the baseline is provided by item-based average ratings  [5]  or user-based average ratings  [22]. The goal here is to predict based on the average ratings provided by users in U S ∪ U T for items in I S ∪ I T. One of the most basic prediction schemes is the ItemAverage scheme where we predict that each item will be rated as the average over all users who rated that item  [5]. Note that though this technique gives a very good estimate of the actual rating but it is not personalized due to same predictions for all the users. We compare X-Map with these three other systems namely: Item-based-kNN, RemoteUser and ItemAverage. Evaluation scheme. We partition the set of common users who rated both movies and books into training and test sets.
Furthermore, we observe in Figure  9 (a) that the user-based models show more improvement than the itembased ones. This behaviour occurs as the item similarities are more static than the user similarities  [19]. Impact of sparsity. We now evaluate how X-Map performs when the size of the training profile of a user, in the target domain, increases from a minimum of 0 (cold-start situation) to a maximum of 6 (low sparsity), in addition to her profile in the source domain  13. This experiment also highlights the performance of X-Map when the sparsity of the dataset decreases. Additionally, we evaluate the accuracy improvement of X-Map over a single domain solution, item-based kNN in the target domain denoted by KNN-sd, as well as over a heterogeneous solution, item-based kNN in the aggregated domain denoted by KNN-cd.
However, web agents are considered a threat to users' privacy as users' data over different e-commerce applications are stored in a central database administered by the web agent. Cross-domain Mediation. Berkovsky et al. [6]  proposed the idea of cross-domain mediation to compute recommendations by aggregating data from several recommenders. We showed empirically that X-Map outperforms cross-domain mediation in Figures  8  and 9. In contrast, X-Map introduces a new trend in heterogeneous personalization in the sense that the user profile from a source domain is leveraged to generate an artificial Al-terEgo profile in a target domain. The AlterEgo profiles can even be exchanged between e-commerce companies like Netflix, Last. fm thanks to the privacy guarantee in X-Map. Merging preferences.
Albeit being model-based and representing an improvement over previous methods, the statistical approaches used by Bowen and colleagues do not use Kriging methods (and mixed models a fortiori). Kriging has been reintroduced in later works (e. g. Terzer et al. , 2013; Van der Veer et al. , 2009) , but the Kriging is done without using mixed models explicitly. Here, we explain how to build isoscapes using mixed models. Our methodology is general and can handle different datasets, isotopes, or environmental materials. As a study case, we will build the isoscape for the hydrogen composition (measured in δ 2 H) of precipitation water using data from the Global Network for Isotopes in Precipitation (IAEA/  WMO, 2017). This will illustrate how to fit the mixed models underlying isoscapes, how to build different isoscapes from the fitted models, how to weight monthly predictions by precipitation amounts to obtain the traditional annual weighted predictions, and how to compare different statistical models fitted on the same data.
We will also show that, at least for the data we use, our approach is more accurate than the widely used approach from  Bowen and Wilkinson (2002)  and  Bowen and Revenaugh (2003). Finally, we will discuss the benefits and limits of the mixed-model based approach with respect to practical applications. We implement the analysis in the free open-source statistical software R (R Core Team, 2017) and perform most computations using the package spaMM  (Rousset and Ferdy, 2014). This package is a multi-purpose package that allows the user to fit mixed models with (but also without) spatial autocorrelation by maximum (or restricted maximum) likelihood. We provide all the source code necessary to produce all the figures and tables present in this paper under the form of a small package (available in Supplementary material).
We developed this package for reproducibility purposes only, and we did not design this package for general usage on other datasets or applications. For general usage we recommend instead to either use spaMM directly, or to use the IsoriX package which is a wrapper for spaMM specifically developed for modeling the spatial distribution of isotopes, as well as to assign organisms to their origin location  (Courtiol et al. , 2016). We assume little prior knowledge of mixed models in this paper and we define some key concepts related to these models in the glossary. We assume nonetheless knowledge of the basic concepts generally introduced in the context of generalized linear models (GLMs) such as linear predictor, regression coefficients, or residuals; otherwise see  Venables and Ripley (2002, Chapter 7)  for an introduction to GLM and  Diggle et al.
The dataset For demonstration, we will fit the aforementioned mixed models on monthly measurements of the isotope composition of the hydrogen (technically expressed as δ 2 H) found in precipitation water. The dataset was provided by the Global Network for Isotopes in Precipitation (IAEA/WMO, 2017), which distributes freely isotope measurements from precipitation water collected all over the world. The coverage of the GNIP data is variable depending on the world region, and we will restrict our analysis to Europe (here defined as locations within the latitude 30°00'N -70°00'N and the longitude 30°00'W -60°00'E), which is a particularly data dense area. We discarded observations for which required information was missing, those corresponding to a month and monitoring station only observed a single time, and those for which "month" samples have been derived from a precipitation collection based on 25 days or less, or on 35 days or more.
We also excluded the data from one monitoring station for which sampled names indicated they were half month samples instead of monthly ones. After this filtering procedure, our data yielded a total of 30546 monthly measurements associated with 326 different locations. These data cover 658 out of the 660 possible months covering the study period. The raw data provided by the GNIP project are readily aggregated into monthly measurements for each available year and sampling location. We chose to aggregate these data further, across years, so to obtain a single mean and variance estimate for each available month-location combination. We aggregated the data this way for faster fitting of both the mean model and the residual dispersion model, as detailed below. We denote the final aggregated dataset GNIPdataEU (table  1 ).
Fitting approach We now want to fit the mixed models on the GNIPdataEU dataset to obtain estimates for the fixed effects β 0 , β lat , β elev and the random effect parameters λ RS , ρ, ν, λ RU of the mean model (eq. 1), and likewise the estimates for the fixed and random effect parameters β D 0 , and λ D RS , ρ D , ν D , λ D.
Fitting the residual dispersion model We first use the spatial mixed-effect model for the expectation of the residuals of the mean model (eq. 6), and fit this residual dispersion model to the variance estimates variso, derived from nyear g monthly observations in each location g (see table  1 ). From our earlier assumption that Error gi is normally distributed with variance φ g , variso is Gamma-distributed with mean φ g and variance 2φ 2  g /(nyear g -1). 2 We can therefore model variso using a Gamma generalized linear mixed-effects model (Gamma GLMM). To account for the variance 2φ 2 g /(nyear g -1) of variso in the residual dispersion model, we need to consider how residual variance is represented in a Gamma GLMM. The package spaMM follows the established usage for Gamma  GLMs (McCullagh and Nelder, 1989) , where the Gamma-distributed response is by default parametrized by its mean µ and a dispersion parameter Φ such that the variance of the response is µ 2 Φ.
In addition, known prior weights w i can be specified, such that the dispersion for each level i of the response variable is Φ i = Φ/w i. We can therefore represent 2φ 2 g /(nyear g -1) as µ 2 φ D /w g for µ = φ g , φ D = 2, and prior weights w g = nyear g -1 for each location g. We now use this result to fit the residual dispersion model on the data for January using the function fitme() from spaMM, as follow: dispfit1 <-fitme( formula = variso ~1 + Matern(1|long + lat) + (1|stationID), family = Gamma(link = log), data = GNIPdataEU1, fixed = list(phi = 2), prior. weights = nyear -1, control. dist = list(dist. method = "Earth"), method = "REML") This simple call produces an R object denoted dispfit1 that contains estimates of the parameters of the residual dispersion model. The arguments formula and family are used to specify the residual dispersion model.
Specifically, in the formula, the "1" indicates to fit one intercept, the term Matern(1|long + lat) indicates to fit a spatially-structured random effect following a Matérn correlation structure where the variable long and lat 3  will be used for the computation of the distance between observations, and the term (1|stationID) indicates to fit an uncorrelated random effect whose realizations will be identical for all observations with the same value for the variable stationID. The argument family specifies the Gamma family with log link, where the link corresponds to the exponential in eq. 6. The argument data indicates in which R object the dataset is stored. The argument fixed is used to fix parameter values(s) in the model, and we use it here specifically to fix the residual dispersion φ D = 2 in dispfit1.
Fitting the mean model The second step of our procedure is to fit the mean model. This is a LMM with two distinctive features. First, we need the predictions φg of the expected square of the residual error in each location, given by the fit of the residual dispersion model. We can derive the predictions from dispfit1 with the function predict() from spaMM and store them within our working dataset with the call: GNIPdataEU1$disp <-predict(dispfit1, newdata = GNIPdataEU1)[, 1] Second, since the fit of the main model is a fit to the mean of nyear g observations in each location g, we will also specify that the residual variance for each such mean is φg /nyear by using prior weights nyear. We fit the mean model for January using the function fitme() as before: meanfit1 <-fitme( formula = meaniso ~1 + lat + elev + Matern(1|long + lat) + (1|stationID), family = gaussian(link = identity), data = GNIPdataEU1, resid.
The dataset for predictions Because it is not possible to make prediction for an infinite number of locations, we will predict the isotope composition on a regular array within the extent of latitudes and longitudes of our selected area. We call these locations prediction locations. The prediction locations are thus arranged in rows of equal latitudes and columns of equal longitudes forming a regular geographic array or rasters. To compute a prediction for each prediction location, we need the parameter estimates from the fit of the mean model and from the fit of the residual dispersion model provided by meanfit1 and dispfit1, respectively. The objects meanfit1 and dispfit1 also contain all necessary parameters required to compute the prediction and residual variances. For each prediction location, we also need the values for all covariates used in the mixed models.
Predicting the isoscapes for January Using the table datapred and the parameter estimates obtained from dispfit1, we start by predicting the residual dispersion variance for each prediction location, according to φg = exp Fix D g + RandomSpatial D g. ( 7 ) Hat symbols denote estimations of fixed effects, predictions of random effects, and functions of them. On the right-hand side of the equation, they thus denote estimations and predictions from the fit of the dispersion model. Note that the uncorrelated random effect is not apparent because the prediction of an uncorrelated random effect is necessarily zero for any new location. This computation is done by a first call to the spaMM function predict(). We directly store the predictions for the residual variance in datapred. In R language, this gives: datapred$disp <-predict(dispfit1, newdata = datapred)[, 1] We can then use the table datapred and the parameter estimates obtained from meanfit1 to compute the predicted isotope value over all possible observations i at a location g as îso g.
= β0 + βlat × lat. abs g + βelev × elev g + RandomSpatial g. (  8 ) Again, the uncorrelated random effect is not apparent because its prediction is zero for any new locations ( RandomUncorr g = 0). This computation is done by a second call to the spaMM function predict(): predict1 <-predict(meanfit1, newdata = datapred, variances = list(predVar = TRUE, residVar = TRUE)) In addition to the point prediction for each prediction location, this second call to predict uses the information obtained from the first call (i. e. datapred$disp) to compute the prediction variance of the mean model. We finally extract the point predictions, the prediction variance and the residual variance, and store them within a single R object denoted pred1 which also contains the prediction locations (see table  3 for content):  pred1 <-datapred[, c("lat", "long")] pred1$pred <-predict1[, 1] pred1$predVar <-attr(predict1, "predVar") pred1$residVar <-attr(predict1, "residVar") Once we have the point prediction, the prediction variance and residual variance for each prediction location, we can then turn the isoscape data we want to represent (pred, predVar or residVar) into a raster and plot the raster for visualization using an R function dedicated for plotting rasters, such as levelplot() from the package rasterVis (see Supplementary material for R code).
It is also possible to assign a different weight to each monthly isoscape before combining them. As an illustration, we will compute the annual average weighted by the estimated amount of precipitation that fell in each month at each prediction location. We chose this example because the use of such annual average weighted by precipitation amounts is common in the isoscape literature  (Bowen, 2010). For this we need to derive weight for each month from precipitation data at each prediction location. These precipitation data can be obtained freely from WorldClim 1. 4  (Hijmans et al. , 2005) , which provides such information as a set of 12 rasters (one for each month) at a resolution of ca. one value per square kilometre. After downloading these rasters, we use the function extract() from the package raster to extract the precipitation amount for the 12 months at each prediction location.
Leave-one-out cross-validation A general method to assess the predictive power of any model is to perform a cross-validation procedure where the model is used to predict data that have been observed but that have not been considered for fitting the model. When a single dataset is available, it is possible to apply a so-called leave-one-out cross-validation. In this case, one observation is excluded from the dataset (called out-of-bag observation) before fitting the model, then a prediction is generated by taking the predictors values attached to the out-of-bag observation, and this prediction is finally compared to the actual observation. This last operation is repeated by considering in turn all observations as out-of-bag observations and a metric is used to measure how close out-ofbag predictions are from their corresponding actual values (e.
An example for model comparison As an illustration, we use our data from January (i. e. GNIPdataEU1) and compare the ability for several pairs of fitted mean and residual dispersion models to predict accurately the observed mean isotope values. We will do so using both the leave-one-out cross-validation and the cAIC. We consider three possible fixed-effect structures for the mean model and for the residual dispersion model. Specifically, we consider a fixed-effect structure defined by i) an intercept only; ii) an intercept, a linear effect of the latitude and a linear effect of the elevation; or iii) an intercept, both a linear and a quadratic effect of the latitude and a linear effect of the elevation. For the interceptonly models, we also consider two alternative random-effect structures.
We repeat these five steps for each station. When this is done, we then compute the root of the average of the squared differences we obtained, as well as the average of the absolute differences, over all leave-one-out samples. We call these two averages the Root Mean Squared Error (RMSE) and the Mean Absolute Error (MAE), respectively. The smaller these error terms are, the better the predictive power. To compute the information criterion, we fit the 16 pairs of models aforementioned to the entire dataset GNIPdataEU1. We then use the cAIC estimate provided by the function AIC from the package spaMM for the mean fit. Because this cAIC value is computed given the predictions of the residual dispersion fit, we need to penalized it by adding twice the number of parameters of the latter fit to the initial cAIC value (see eq.
The fitted models To illustrate how to model the spatial distribution in isotope composition using mixed-models, we have fitted the mean model meanfit1 and the residual dispersion model dispfit1 on a aggregated dataset containing the estimated mean and inter-annual variance in δ 2 H found in precipitation water across Europe for the month of January (GNIPdataEU1). In this first section, we explain how to interpret the estimates obtained for such models.
Mixed model vs the BWR method We have compared our mixed model approach to one of the most widely used approach in the field: the BWR method  (Bowen and Revenaugh, 2003; Bowen and Wilkinson, 2002). We have detailed theoretical reasons for the mixed model approach to be better. Accordingly, we found that, at least on the dataset we used, the mixed models approach predicts better unobserved isotope measurements. The mixed model approach is much slower to fit than the BWR fitting procedure. In our example, the BWR fitting procedure was up to 3 orders of magnitude faster than the mixed model approach. This is however not a major practical limitation since the aggregation scheme we propose still allows for a complete fit in a few hours maximum on a personal computer. Most users are likely to rely on dataset smaller than the one we used, bringing the time cost down.
g. the one used by  Van der Veer et al. , 2009). Third, our approach also makes specific assumptions about the random effects. We have considered all random effects to be normally distributed. It is easy to consider alternative distributions (e. g. Gamma) for the random effects whose realizations are not spatially autocorrelated (here RandomUncorr & RandomUncorr D ) and the package spaMM we used to fit the mixed models offers this possibility. However, because random effects that present autocorrelated random effects must be described in terms of mean and covariances, using alternative distributions is not currently possible for the spatial random effect (unless one considers other elliptical distributions closely related to the Gaussian;  Embrechts et al. , 2002). Other Kriging approaches for which assumptions have been made explicit also assume normality  (Van der Veer et al.
Conclusion The application of likelihood methods for mixed models to build and study isoscapes has been under-developed so far. In this paper we have introduced this approach to unfamiliar readers, but we have also detailed lesser known aspects and unusual adjustments of these methods to the present problems. As we have shown, our approach can be used to analysed the very large public dataset of the isotope composition of precipitation water provided by the Global Network for Isotopes in Precipitation (IAEA/WMO, 2017), but it can also be used to analyse other sources of data (e. g. the isotope composition of oceanic water data). For easier access to the methods, we have provided both a small R package called ModellingIsoscapesUsingMixedModels (included as Supplementary material) illustrating how to code the approach using the more general package spaMM, and the package IsoriX  (Courtiol et al.