dataesr/scientific-paragraphs-categorization huggingface dataset:
We used the well-established DA system designed for ORCHIDEE (ORCHIDAS: https://orchidas. lsce. ipsl. fr) (Kuppel et al. , 2014; MacBean et al. , 2018; Peylin et al. , 2016) , in which a cost function that represents the misfit between the model and the data -taking into account uncertainty in both -is iteratively minimized using the genetic algorithm (GA; see Methods and Data). Beyond investigating if the DA system could account for model-data discrepancies in dyland NEE simulations, our second objective was to identify which parameters (therefore, which processes) may be responsible for model errors. To address this objective, we performed multiple optimization tests with combinations of parameters related to different model processes in order to identify which processes were most influential in improving the model mean annual NEE and IAV.
We focused in particular on which processes are responsible for model failure to capture NEE IAV. Our focus on improving NEE IAV was partly because of the dominant role dryland ecosystems are thought to play in controlling global C cycle IAV, and partly because we expected that, with the exception of sites that are a strong C sink, eddy covariance estimates of mean annual NEE may be impacted by uncertainties in CO 2 flux partitioning. The methods and data are described in Section 2 and the results are presented in Section 3 and discussed in Section 4.
Flux Tower Measurements At all 12 SW US sites, flux tower instruments collect 30-min measurements of meteorological forcing data and eddy covariance measurements of net surface energy and carbon exchanges, which are available from the AmeriFlux data portal (http://ameriflux. lbl. gov). Meteorological forcing data included air temperature and surface pressure, precipitation, incoming long and shortwave radiation, wind speed, and specific humidity. To run the ORCHIDEE model, we partitioned the in-situ precipitation into rain and snowfall using a temperature threshold of 0°C. The site-level meteorological forcing data were gap filled utilizing downscaled and corrected ERA-Interim data following the approach of  Vuichard and Papale (2015). Gross primary productivity (GPP) and the ecosystem respiration (R eco ) were estimated from the net ecosystem exchange (NEE) via the flux partitioning method described in  Biederman et al.
Parameters Optimized The full set of parameters included in the assimilations optimized are described in Table  S1  with their prior values, prior uncertainty, and upper and lower bounds for different PFTs based on literature analysis, parameter databases and expert knowledge of the model equations. Prior values are the default parameter values used in all non-optimized ORCHIDEE simulations. In the most past ORCHIDAS studies with previous versions of ORCHIDEE, only subsets of ORCHIDEE C cycle parameters have been optimized  (Bastrikov et al. , 2018; Kuppel et al. , 2012 Kuppel et al. , , 2014;; MacBean et al. , 2015 MacBean et al. , , 2018;; Santaren et al. , 2007; Verbeeck et al. , 2011). In this study, we considered all possible C cycle related ORCHIDEE parameters to fully explore all sources of parameter uncertainty that is contributing to uncertainties in modeled net and gross CO 2 fluxes.
The impact of the temporal validation tests is presented in Section 3. 1 and 3. 2. All other model evaluation metrics refer to the assimilation time window. We also evaluated the posterior parameter values using the limited available trait data close to the sites from the TRY database  (Kattge et al. , 2020). We searched within 0. 5° of the latitude and longitude of each site for trait data related to the species present at each site. We found one estimate of specific leaf area (SLA) for pinus ponderosa close to US-Fuf, and maximum rate of rubisco activity-limited carboxylation at 25°C (V c,max ) and leaf longevity estimates for larrea tridentata (creosote shrubs) close to US-Ses. The comparison of these trait values to posterior parameter estimates is presented in Section 3. 3.
Impact of Optimization of all Parameters (P1) on Model Net and Gross CO 2 Fluxes Across all sites, the prior ORCHIDEE simulations (i. e. , before parameter optimization) fail to capture both the mean annual NEE at mean C sink and source sites and the NEE IAV across all sites (Figure  1a ) -as also seen for all TRENDY TBMs in  MacBean et al. (2021). Across all sites, optimizing all C cycle-related parameters (phenology, photosynthesis and post C uptake -assimilation scenario P1) with NEE data dramatically increases the ability of the model to capture both the mean C source/sink behavior and the IAV (Figure  1b ). C sink and source sites show significant improvement in terms of both mean annual NEE and IAV. At the pivot sites there was no strong bias in NEE with either the prior or posterior parameters (given their mean annual NEE is close to zero); therefore the optimisation has impacted the simulation of NEE IAV rather than the mean annual NEE (as represented by the correlation and slope values shown in inset figures in Figures  1a  and 1b ).
We compared the posterior parameter values for all relevant assimilation scenarios to the available trait data from the TRY database (Figure  S13 ). The two photosynthesis traits (SLA for pinus ponderosa close to US-Fuf and V c,max for larrea tridentata close to US-Ses were overand underestimated by the posterior values across all assimilation scenarios, respectively, whereas the leaf longevity for larrea tridentata measured close to US-Ses was well captured by the P3 assimilation scenario. There is not enough trait information to perform a rigorous validation of the posterior parameter values. The existing measurements may differ from the model due to the fact that the traits were not measured at the same location. However, mismatches between the posterior parameter values and traits presented here highlights that we need to collect more trait data with which to evaluate the optimized parameters, in addition to using the DA framework to explore how parameters may vary over space and time.
Further Testing and Developments Needed to Improve Modeling of Dryland C Cycling In this study, we have shown that it is possible to account for model discrepancies in both the mean annual NEE and NEE IAV at a range of semi-arid SW US sites via optimization of C cycle parameters within a Bayesian DA framework. We used weak prior constraints (i. e. , large prior parameter bounds) to give the assimilation the maximum chance to correct any model errors. Our goal was not to identify the ideal "correct" set of C cycle parameters for capturing semi-arid vegetation and C cycle dynamics, but rather to identify whether, within the current model representation, we could account for model-data mismatches. Looking at the individual parameter plots for the P1 assimilation scenario (Figure  S14 ), we find that at some sites several posterior parameters are "edge-hitting" (e.
Here, we aimed to facilitate our understanding of which processes may be responsible for errors in modeling of semi-arid C fluxes by using the different assimilation scenarios as tests of which parameter sets (and therefore, which processes) most improve the model-data mismatch. The assimilation with all C cycle and vegetation parameters (P1) performed the best in terms of correcting underestimates in modeled mean annual NEE and IAV. However, the additional assimilation scenarios (P2 to P7) further demonstrated that phenology parameters are likely key for improving semi-arid ecosystem NEE IAV. Issues with semi-arid phenology in TBMs have been documented elsewhere  (Dahlin et al. , 2015; MacBean et al. , 2015; Renwick et al. , 2019; Teckentrup et al. , 2021 in review; Traore et al. , 2014; Whitley et al.
, 2004). However, we note that as we include multiple data streams in the assimilation the computational cost will increase; therefore, there will be a need for sensitivity analyses to select only the parameters to which assimilated variables are most sensitive. Improving estimates of NEE IAV may require additional terms in the cost-function that specifically estimate the model-data misfit at annual timescales (e. g. ,  Desai, 2010) , as opposed to evaluating the daily timestep as we do in the cost function in this study. We also need to find better ways to estimate the combined observation and model structural errors (R matrix), including assessing observation temporal autocorrelation and error correlations between different datasets in future multiple data stream DA experiments. In future SW US C cycle DA studies we will also consider options for using the available trait information to better estimate prior parameter bounds and off-diagonal elements of the prior B matrix (e.
2). Despite the need for many more DA system tests at dryland sites, the assimilation experiments presented here already demonstrate that strong reductions in parameter uncertainty and dramatic improvements in model-data fit are possible using in situ dryland CO 2 fluxes. Our results clearly show that, in addition to model developments that may be needed for models to better represent dryland ecosystems, C cycle related parameters likely need optimizing by TBM groups before they can accurately model dryland CO 2 fluxes. Only by addressing these issues will we be able to reliably use TBMs to accurately simulate regional to global-scale dryland contributions to IAV and long-term trends in the global C cycle. Funding for AmeriFlux data resources and data collection at US-SRM, US-SRG, US-Wkg, and US-Whs was provided by the U.
S. Department of Energy's Office of Science and the USDA (an equal-opportunity employer). Data collection at sites US-Vcp, US-Vcm, US-Mpj, US-Wjs, US-Seg, and US-Ses were funded by the U. S. Department of Energy EPSCoR (DE-FG02-08ER46506), and the Department of Energy Ameriflux Management Project (Subcontract 7074628), and the Sevilleta Long Term Ecological Research site (NSF-DEB LTER 1440478). The US-Fuf site was supported by grants from the North American Carbon Program/ USDA CREES  NRI (2004-35111-15057 and 2008-35101-19076).
Data Availability Statement The ORCHIDEE model is under a free software license (CeCILL; see http://www. cecill. info/index. en. html) and the source code is visible here: https://forge. ipsl. jussieu. fr/orchidee/browser/tags/ORCHIDEE_2_0 (Peylin et al. , 2021). The ORCHIDEE model code is written in Fortran 90 and is maintained and developed under an SVN version control system at the Institute Pierre Simon Laplace (IPSL) in France. The ORCHI-DAS code is currently in the process of being put on a GitHub repository but for now it is available on request to vladislav. bastrikov@lsce. ipsl. fr. Meteorological forcing data and eddy covariance measurements of net surface energy and carbon exchanges at 30-min intervals are available from the AmeriFlux data portal (http://ameriflux. lbl. gov). The model outputs from ORCHIDEE simulations and post-processing python scripts for manuscript figures and tables are freely available in a Git repository (https://github.
Cloud gaming applications have gained great adoption on the Internet particularly benefiting from the wide availability of broadband access networks. However, they still fail to meet users' quality requirements when accessed using cellular networks due to common wireless channel degradations. Machine Learning (ML) techniques can be leveraged to detect such anomalies during users' cloud gaming sessions. In this respect, unsupervised ML approaches are particularly interesting since they do not require labeled datasets. In this work, we investigate these approaches to understand their performance and their robustness. Our dataset consists of game sessions played on the public Google Stadia Cloud Gaming servers. The game sessions are played using a 4G network emulation replicating the capacity variations sampled on a commercial 4G network.
This paper aims at addressing some of these questions. Specifically, it presents an evaluation of unsupervised machine learning approaches for the detection of cloud gaming performance degradation. A particular emphasis of our study is on the detection of anomalies based on multivariate time-series KPIs datasets of cloud gaming sessions under different 4G network conditions. The game sessions are played on the public Google Stadia platform and the game traffic is routed through the Internet to be processed by Stadia's servers. The 4G network conditions are emulated with the Mahimahi tool  [3] , based on real cellular network conditions from a commercial french ISP. We assess the ability of the models to efficiently detect anomalous behaviours as well as their robustness against data contamination.
Unsupervised ML approaches for anomaly detection assume that the data used for training is free from anomalies and contain only normal instances. However, the data collected in real-life scenarios is rarely clean and it can be very cumbersome to remove the anomalous samples from the training data. In this paper, we also show how different data splitting strategies and evaluation procedures can impact the achievable results. The remainder of this paper is organised as follows. Section II discusses previous works on cloud gaming performance analysis studies and unsupervised ML methods for anomaly detection. Section III presents the experimental protocol used to collect the datasets while Section IV describes our methodology for the evaluation of the ML algorithms. We present the evaluation results in Section V and conclude the paper by outlining future work in Section VI.
The anomalous samples are those that lie beyond that hypersphere. Due to their performance on different learning tasks, the methods that have more interest in the literature are those based on neural networks and deep-learning methods. Recent methods that obtain state-of-the-art results on benchmark time-series datasets are based on Auto-Encoders (AE). Auto-Encoders learn how to reconstruct input data from a lowdimensional representation (referred as latent variable) by minimizing the reconstruction error and use an anomaly score (that is the reconstruction error) to identify anomalies. Many variants of Auto-Encoders were developed for anomaly detec-tion. For instance, the LSTM-VAE  [19]  uses a LSTM (Long Short-Term Memory) neural network architecture, to model temporal dependencies in the data, and a VAE (Variational Auto-Encoder) to estimate the latent variable and the distribution of the data.
The anomalous samples are those with a low log-likelihood. DAGMM (Deep Auto-encoder Gaussian Mixture Model)  [20]  model combines an auto-encoder with a Gaussian mixture model and then can model the probability distribution of the output samples. OmniAnommaly  [21]  uses a VAE, a planar NF (Normalizing Flows), a Gaussian state model and a GRU (Gated Recurrent Units which is a LSTM variant) to learn temporal dependencies and stochasticity of the multivariate time-series for anomaly detection. The methods discussed above were applied to detect anomalies based on time-series datasets for different application domains (industry, server machines, engines, web applications. ). To the best of our knowledge, none of these works evaluated anomaly detection in cloud gaming sessions which is the focus of this paper.
By recording the timestamps of the packets sent and the acknowledgements received, we can generate the uplink and downlink txops files where each line matches a time in milliseconds when the real base station can deliver a MTU-sized packet. The new txops files are generated on 6 different conditions (5 static ones and one in a mobility scenario on the highway) on Orange 4G network in Lannion (France). The static conditions differ in the mean downlink throughput that can be achieved (from 40 Mbps to 220 Mbps). The txops files data and further details of this experiment are available on this website  8.
B. Testbed and data collection With the txops files collected, we perform the measurements of the CG time-series KPIs. We set up a testbed, depicted in Fig. 1 , consisting of a Windows 10 laptop client (where the games are played through the Chromium desktop) and a Ubuntu 20. 04 laptop as a network emulator (where the Mahimahi platform is installed). The latter can access the cloud gaming servers through a FTTH (Fiber-To-The-Home) Internet connection to avoid shifting the bottleneck from the network emulator to the WAN interface. Thereby, during the game sessions, the incoming uplink/downlink packets in the network emulator are queued and released by the LinkShell tool, in accordance with the transmission opportunities files representing the behavior of the 4G collected network conditions.
A. Problem formulation According to Chandola et al. [13] , anomalies (or outliers) can be classified into three classes: point anomalies (individual anomalous observations), contextual anomalies (observations considered as anomalous in a specific context) and collective anomalies (group of anomalous observations). In this work, we focus on point anomalies in cloud gaming sessions and leave the other classes of anomalies for future work. To detect anomalies, we use unlabelled KPIs datasets composed of multivariate time-series which are sequences of data observations x = {x 1 , x 2 ,. , x T } where T is the length of x, and an observation x t ∈ R m corresponds to a m-dimensional vector at the time t. Unsupervised Learning anomaly detection task on multivariate data consists of learning a model given the set of normal observations x and assigning for each unseen observation xt a binary variable y t ∈ {0; 1} based on a anomaly score that measures how much the unseen observation differs from the set of normal observations.
B. Data processing and splitting strategies The features of our datasets are the DECAF metrics and ours related to the 4G emulated network. The DECAF metrics are logged and stored on the instrumented Chromium, each time a video frame finishes the frame processing pipeline stage in WebRTC. Since we include our own metrics which do not have the same temporal alignment, we need to resample the data to have a fixed time-step of 5ms between each observation. The features of the datasets are normalized by removing the mean and scaling to unit variance with Standard Scaler 11 before the training step. To obtain the training set and test set, we use the same methodology as Zhong et al. [20] : all the data collected are merged and split into train/test as follows. The training set contains 50% of the normal samples of the whole dataset and the remaining 50% are in the test set.
The test set additionally includes p = 60% of the anomalous samples and the other (1 -p)% are in a set called contamination set. In the experiments, δ% (with δ ∈ {0, 5, 10, 20, 50, 100}) of the data coming from this contamination set are fed back into the training set to identify how the presence of anomalous samples can impact the training and the performance of the models. We use this training strategy by assuming that normal samples in cloud gaming sessions can also be collected in low throughput network conditions. The models then, learn to detect anomalous sessions even on 4G network conditions with lower throughput. In the rest of the paper, this data splitting strategy is referred to as the mixed-dataset strategy. We also choose a different data splitting strategy to evaluate its impact on model performance.
We assume that normal observations in cloud-gaming session occur only at higher downlink bitrates (> 100 Mbps). The data collected in the networks conditions with an average downlink throughput higher than 100 Mbps are used as the training set and all 9 https://support. google. com/stadia/answer/9607891?hl=fr/ 10 https://www. nvidia. com/en-us/geforce/products/geforce-now/system-reqs/ 11 https://scikit-learn. org/stable/modules/generated/sklearn. preprocessing. StandardScaler. html/ the data coming from the other conditions as the test set. Anomalous samples from the training set are removed and used as the contamination set from which some samples are fed back into the training set in the same way as mentioned previously. The issue with this strategy is that the datasets are severely imbalanced: the data used for the test set contains many anomalous samples and the evaluation of the models on this test set does not allow to assess the performance on observations coming from the "best" networks conditions.
The base implementations of PCA, IForest and OC-SVM are from Scikit-Learn while the Auto-Encoder and LSTM-VAE are from our custom implementation with PyTorch. The details, the code and the hyper-parameters values of our implementation are available on Github 14. For each model, we perform 5 training runs, each with one of the two aforementioned data splitting strategies. This allows us to report reliable results since the induced variability of the random sampling in the splitting and the parameters initialization is taken into consideration.
A. Performance with the mixed-dataset Table  II  reports the precision, recall and F1-score of the five models included in our study, with the different anomaly contamination ratio δ on the mixed-datasets. The F1-score is a harmonic mean of the Precision and the Recall. The Precision assess the ability of the model to precisely detect the anomalies and the Recall its ability to detect all the anomalies. The best model will therefore have higher Precision and Recall scores. All the models have their best F1-score with no contamination (δ = 0) except for the iForest model. The AE and LSTM-VAE models have the best score (F1-score = 88%) and this results from high Precision (= 98%) and Recall (= 80%) values. We can have high confidence on the detected anomalies but due to the lower Recall value, some of the anomalies of the test set may be missed by these models.
The iForest model has the worst F1 score when there is no contamination (F1= 1%) due to a lower Recall. In fact, the model does not have the ability to isolate the anomalous samples since it assumes that there are few anomalies, and they 14 https://github. com/joelromanky/cg-ano-detect-eval Nevertheless, the results of all the models, except the iForest, drop with the increase of the contamination ratio. The performance of the AE and LSTM-VAE plummets as soon as anomalies are injected (from 88% to 1%) while the results of the PCA remain low and decrease as the contamination ratio increases. The bad results of the PCA, AE and LSTM-VAE could be explained by the fact that those models need only normal samples to be efficiently trained. With the increase of anomalous samples, the models can not detect all of them in the test set.
For instance the AUC scores for the AE are from 95% to 56% while the F1-scores are from 88% to 2%. This shows how misleading the AUC score can be even if our test set is not too much imbalanced (58% of anomaly ratio) and confirms the conclusions drawn in  [25]  about the use of AUC with imbalanced datasets. Apart from the performance of the models on the anomaly detection tasks, we also evaluate the training time of each model in a given configuration. These results are depicted in  Take-away: The best models without data contamination are the AE and LSTM-VAE but they show poorer performance with data contamination. Since in real-life situations, the collected data always contain anomalies, the OC-SVM or the iForest should be preferred although the OC-SVM has a longer training time.