dataesr/scientific-paragraphs-categorization huggingface dataset:
25, 1, 4}), which makes it easier to tuneour unsupervised loss serves as a consistency regularization. Due to space limitations, this analysis is available in appendix D. Adding our masking strategies on top of DBS has a significant impact on all datasets, with the unigram variant being up about 2 points over the vanilla DBS on average. On all datasets except Clinic, given only 10 labeled samples per class (low profile), it even outperforms the supervised baseline which is given the full training data (full profile). This means that PROTAUGMENT does better than prototypical networks with much less -15 times, and up to 47 times, depending on the dataset -labeled sentences per class. Those results indicate that our method more than compensates for the lack of labeled data and that no matter the amount of data available for the training class, there is a performance ceiling you cannot overcome without adding unsupervised knowledge from the validation and test classes.
In the full profile, when given all the training data, our method greatly surpasses the Prototypical Network -3. 58 points given 1 shot, on average. Moreover, PROTAUGMENT is not only suited for the case where very little training data is available (low profile): when sampling shots from the entire training dataset (full profile), it outperforms a fully supervised baseline. Furthermore, note that our method is consistently more stable than the supervised baselines, as its average standard deviation over the different runs is much lower than the vanilla Prototypical Network.
Datasets Accuracy stats Data Profile Method Banking HWU Liu Clinic (AV G ± ST D) Table  4 : 5-way 1-shots and 5-way 5-shots accuracy on the test sets for each dataset. The ours method is PROTAUG-MENT (unsupervised consistency loss using diverse paraphrases) equipped with different paraphrasing strategies. K = 1 K = 5 K = 1 K = 5 K = 1 K = 5 K = 1 K = 5 K = 1 K = 5 For each dataset × C-way K-shot setting, we compute the average and the standard deviation over the 5 runs (see Section 5. 2), so that the last two columns contains average accuracy and ± the average standard deviations. For each data profile, we highlight the best method in bold. We underline the methods on the low profile which perform better than the Prototypical Networks on the full profile. We trained 400 different meta-learners -5 methods, 2 data profiles, 4 datasets, 2 meta-learning setup (K = 1, 5) and 5 runs for each configuration.
Masking strategies We experimented with three variants of the unigram strategy (Section 4. 2), each assigning a different drop chance to each token depending on its position in the input sentence. In our experiments, we did not observe any significant difference in performance when putting more weight on the first tokens (down), or last tokens (up), or the same weight on all tokens (flat) (Detailed results in appendix C). We also conducted experiments where we tune the value p mask , from 0 to 1, selecting 0. 7 as the best trade-off (Figure  2 ). This figure also clearly shows that the Clinic dataset is one order of magnitude easier to solve than the other datasets.
Conclusion In this work, we proposed PROTAUGMENT, an architecture for meta-learning for the problem of classifying user-generated short-texts (intents). We first introduced an unsupervised paraphrasing consistency loss in the prototypical network's framework to improve its representational power. Then, while the recent diverse beam search algorithm was designed to enforce diversity between the generated paraphrases, it does not ensure diversity between the generated paraphrases and the original sentences. To make up for the latter, we introduce constraints in the diverse beam search generation, further increasing the diversity. Our thorough evaluation demonstrates that PROTAUGMENT offers a significant leap in accuracy for the most recent and  challenging datasets. PROTAUGMENT vastly outperforms prototypical networks, which was found to be the best meta-learning framework for shorttexts  (Dopierre et al.
, 2021)  against unsupervisedextended Prototypical Networks  (Ren et al. , 2018) , Matching Networks  (Vinyals et al. , 2016) , Relation Networks  (Sung et al. , 2018) , and Induction Networks  (Geng et al. , 2019) , thereby making PRO-TAUGMENT the new state-of-the-art for this task. We provide the source code of PROTAUGMENT as well as code for evaluations reported in this paper on a public repository 4   orig: what's the recipe for fish soup back: What is the recipe for fish soup dbs 0: How do you make fish soup? How is the recipe determined? dsb 1: How can you recipe for fish-sugary food? dbs 2: What are the recipes for Fish soup and how is it prepared? orig: Find easy recipe for almond milk back: Find an easy recipe for almond milk dbs 0: What are some good recipe for Almond milk? dsb 1: What are some good ways of making Almond milk? dbs 2: How do I make Almond milk for a beginner? K = 1 K = 5 K = 1 K = 5 K = 1 K = 5 K = 1 K = 5 K = 1 K = 5.
Figure1: PROTAUGMENT illustrated on a 3-way 2-shot short text classification meta-learning task (C = 3, K = 2). BART is pre-trained for the paraphrasing task on three datasets: Quora (Sharma et al. , 2019) , MSR (Zhao and Wang, 2010)  and Google PAWS-Wiki (Yang et al. , 2019; Zhang et al. , 2019). The paraphrase model is used to paraphrase unlabeled samples but equipped with diversity strategies (back translation being proposed as a baseline). The final loss is computed using a loss annealing scheduler, which is expected to smooth the supervised (given shots) and unsupervised (augmented unlabeled sentences) prediction errors to yield parameter gradients. A new episode means sampling other classes along with their support and query points.
). HWU64 HWU64 (Xingkun Liu and Rieser, 2019) classifies 25, 716 user utterances with 64 user intents. It features intents spanning across 21 domains (alarm, audio, audiobook, calendar, cooking, datetime,. ). When separating training, validation, and test labels, we ensure each domain is rep- Dataset #sentences #classes train/valid/test (total) Available sentences/class #tokens/sentence Banking77 13, 083 25/25/27(77) 170 ± 33 11. 7 ± 7. 6 HWU64 11, 036 23/16. 4/24. 6(64) 172 ± 40 6. 6 ± 2. 9 Clinic150 22, 500 50/50/50(150) 150 ± 0 8. 5 ± 3. 3 Liu 25, 478 18/18/18(54) 472 ± 831 7. 5 ± 3. 4 Table1: Main statistics of intent detection evaluation datasets. For HWU64, each split's number of classes varies at each run to ensure there is no cross-split domain, hence the decimal number.
The objective of this pre-calibration and evaluation is to provide a solid base for our future work which aims to calibrate the cycling flow per link in MATSim for Paris with the normalized flow from crowdsensing data e. g. GPS traces of Geovelo and the measurements from eco-counters  [6]. This paper is structured as follows: Section II provides a simple introduction on MATSim and the packages used in this study. Section III presents the pre-calibration of DCM and its integration in the simulation. Section IV evaluates cycling behavior. The last section concludes the paper and discusses the future work in more detail.
Île-de-France scenario To generate a synthetic population from raw survey data, we use the pipeline presented in  [1]. The synthesized agents are sampled from the population census 2017 published by INSEE (Insistut National de la Statistique et des Etudes Economiques). According to their socio-demographic attributes such as sex, age and profession, each synthesized agent is associated statistically to an activity chain from the regional Household Travel Survey 2010 (Enquête Globable Transport). An activity chain contains daily activities with duration, connected trips between activities and their travel times. To assign the location of each activity of each synthesized agent, we use the facility census 2017 (Base permanente des équipements) and commuting data from the census 2017. The socio-demographic attributes, activity chains and location assignments are computed via statistical matching such that the synthesized population contains the statistical attributes of the surveys.
The goal of this step is to reproduce the travel pattern observed by the regional Household Travel Survey (HTS) in MATSim without running any time-consuming simulation. Since the modeled cycling behavior could impact strongly the mode choice decision on bike, we evaluate, then, the cycling routing with GPS traces obtained dynamically from the mobile application Geovelo  [5]. The objective of this pre-calibration and evaluation is to provide a solid base for our future work which aims to calibrate the cycling flow per link in MATSim for Paris with the normalized flow from crowdsensing data e. g. GPS traces of Geovelo and the measurements from eco-counters  [6]. This paper is structured as follows: Section II provides a simple introduction on MATSim and the packages used in this study. Section III presents the pre-calibration of DCM and its integration in the simulation.
Parameter Estimation The structure of the utility function   for a transport mode  is as follows: where  is mode-specific constant utility,  are travel variables e. g. travel time,   distance and   speed,   and   indicate respectively if a trip is cross-urban or only inside of Paris. are indicative attributes of agents e. g. driving license,   public transport subscription,  _ car availability and  _ bike availability. The objective of the pre-calibration is to search a parameter vector  which enables the DCM to predict correctly the transport mode for a given trip. The travel variables are estimated from OTP. The quality of the prediction is evaluated by the data of the Household Travel Survey considered as the reference in this study. Note that the model formulation (without cost and with highly correlated variables such as travel time, distance, and speed) is not suited for an econometric interpretation, but, rather, aims at providing a good fit with reference data for a first version of the model.
Parameter Estimation The structure of the utility function   for a transport mode  is as follows: public transport subscription,  _ car availability and  _ bike availability. The objective of the pre-calibration is to search a parameter vector  which enables the DCM to predict correctly the transport mode for a given trip. The travel variables are estimated from OTP. The quality of the prediction is evaluated by the data of the Household Travel Survey considered as the reference in this study. Note that the model formulation (without cost and with highly correlated variables such as travel time, distance, and speed) is not suited for an econometric interpretation, but, rather, aims at providing a good fit with reference data for a first version of the model. A transport mode is selected by choosing the highest utility for a given trip.
As an example in Figure  7 , the trend of the simulated flow at Rue de Rivoli during the day does not follow the measured one, especially in the evening. It is, therefore, valid to ask whether agents are routed to the correct links. The Household Travel Survey 2010 provides less than 2000 cycling trips with origin-destination at departmental level. Furthermore, the few records could have been registered already ten years ago. Yet, the fragmented measurements from the counters also do not provide a flow of the whole cycling network. Therefore, these datasets are not sufficient to render a detailed picture of cycling behavior in Paris. The available data of Geovelo is now newer and more disaggregated than the one of EGT. We notice that the trip-mean cycling speeds follow Gaussian distributions centered at 4km/h, 8km/h and 10km/h for the data of EGT, the simulation, and Geovelo, respectively.
It is, therefore, valid to ask whether agents are routed to the correct links. The Household Travel Survey 2010 provides less than 2000 cycling trips with origin-destination at departmental level. Furthermore, the few records could have been registered already ten years ago. Yet, the fragmented measurements from the counters also do not provide a flow of the whole cycling network. Therefore, these datasets are not sufficient to render a detailed picture of cycling behavior in Paris. The available data of Geovelo is now newer and more disaggregated than the one of EGT. We notice that the trip-mean cycling speeds follow Gaussian distributions centered at 4km/h, 8km/h and 10km/h for the data of EGT, the simulation, and Geovelo, respectively. This difference between the simulation and the observations of Geovelo could be due to the representativeness of the crowdsensing data, the modeled cycling network and the routing.
Conclusion and Future Work In this paper, it was illustrated that the travel patterns observed in the Household Transport Survey are coarsely reproduced by the pre-calibrated DCM in MATSim. The evaluation of the cyclist model with GPS data showed that the cycling routing in our MATSim model for Paris needs to be improved. The objective of this pre-calibration and evaluation is to provide a solid base for our future work which aims to calibrate the cycling flow per link with the normalized flow from crowdsensing data e. g. GPS traces of Geovelo and the measures from eco-counters. For this future work, the approach is two-folded. The first step aims to provide a realistic itinerary to each cycling trip. To this end, the behavioral cyclist model will be adjusted with indicators generated dynamically from Geovelo GPS data such as road quality per link and moving speed per link.
HEP-Workloads HEP-Workloads is a collection of applications provided by several experiments. The repository [14] contains the code and infrastructure, both common and workload-specific, to build a standalone HEP-Workloads container image for each application. Each HEP-Workloads container encapsulates the software and input data needed to run the application of a specific experiment. The software of the HEP experiments is typically stored in the CVMFS file system  [15]. CVMFS is a remote file system, whereas the HEP-workload containers must contain all the software to avoid any dependency on remote services. The benchmark applications typically need a subset of the software stack of an experiment. As a result, a procedure has been developed, based on the CVMFS Trace and Export utilities, to export the application software from CVMFS to a local folder inside a container.
In addition, conventional names are assigned to the parameter set. For example, the HEPscore software is being released with a proof-of-concept configuration named HEPscore , based on the set of workloads in Table  1  with the exclusion of the ATLAS SIM workload. 1  This approach is common also for SPEC CPU 2006: HS06 is the all_cpp. bset [1]  set of applications included in that suite. This is only one of the many possible sets of applications that can run using SPEC CPU2006. A WLCG Task Force  [27] , consisting of members from the global HEP community, was formed in 2020 to evaluate the feasibility of replacing HS06 benchmark with HEPscore benchmark for the WLCG computing resources. The role of the Task Force is to identify the composition for the new benchmark, that is temporarily named HEPscoreX.
Results Using HEPscore Ťhis section describes the use of the HEP Benchmark Suite with the HS06 64bits and HEPscore benchmarks. HEPscore is not the future HEP benchmark as it is based on older software of the LHC experiments and not all the experiments are included. Furthermore, the applications used in the measurements are equally weighted giving a higher significance to the CMS software with its three distinct applications. However, running the Suite to measure HS06 and HEPscore provides valuable feedback on the functionality of the infrastructure and information on the usability of the HEPscore as a benchmark. The benchmarks were run on servers at a number of WLCG and HPC sites used by the HEP community to evaluate a wide range of CPU models. Fifteen different x86 Intel and AMD CPU models have been studied including newly released and older models with physical cores ranging from 16 to 128.
ERA Acute supports a variety of analyses, from simple screening studies based on oil spill statistics and potential impact areas to more in-depth impact and recovery calculations on species and habitats. The ERA Acute software tool has been built to enable and provide ease of use of the methodology and results. Visualizations of impact and risk areas can be made at several levels, from simulations and scenarios to whole cases. Results can have a monthly resolution to show variations throughout the year. This enables a wide range of decision-support from risk screening studies, impact assessments, risk quantification, risk management including effect of mitigating measures (NEBA/SIMA) evaluations to properly inform oil spill response planning. The methodology is suitable for global use and will be the recommended approach for oil spill risk assessments for offshore operators on the NCS.
ERA Acute Usage Areas ERA Acute is an enhanced and globally applicable quantitative oil spill risk assessment methodology (meeting the guidelines set by IPIECA-IOGP 2013) and software tool, and is applicable for various risk assessment purposes, depending on user's need and data availability: 1. Risk screening 2. Input to concept selection/risk comparison 3. Environmental impact assessments 4. Site-specific decision making 5. Risk estimation and evaluation/risk ranking Evaluation and prioritization or risk reducing measures; e. g. ERA Acute can provide quantitative input to NEBA/SIMA in order to inform oil spill response planning (IPIECA-API-IOGP 2017). Questions that can be answered using ERA Acute are: • What is the possible impacted area of a spill scenario? • What are the possible consequences to species and habitats in the various environmental compartments? • Could there be a risk for adverse environmental effects? • What is the probability for different consequences, i.
Without saying anything about the presence of sensitive resources or habitats, and just by analyzing the results from a high number of different oil spill trajectories, the screening process can highlight areas and periods of environmental concern in different environmental compartments. This is denoted level A. 1 (see  Sect. 1. 3. 3. 1). The ERA Acute software tool can visualize important features from the oil spill modelling such as probabilities of oiling above an effect threshold value (i. e. surface oil film thickness or water column concentration), and statistical parameters such as minimum, mean and maximum values of oil volumes and concentrations in geographic areas. All oil spill parameters can be visualized in a map in an ERA Acute screening assessment. The risk screening can as such define the boundaries for further detailed impact and risk assessments, provide insight into the needs for further data gathering or inform concept selection in an early stage without extensive assessment approaches.
Assuming that certain species with specific oil spill sensitivities are present within the spilled area (Level A. 1), the ERA Acute screening can also estimate the expected mortality of such a species in this area (Fig. 2. 1, left). By adding information on the spatial-temporal distribution of the VEC in an area in the form of a data set of presence/non-presence, the first screening of probable impacts and impact areas can be further refined and examined (level A. 2). The area of average potential mortality may for example be narrowed down to a particular seabird species/population in the breeding season within the breeding area (Fig. 2. 1, right) (see  Sect. 1. 3. 3. 1). In addition to the statistical results, visualization of single simulations is a possibility within the ERA Acute software tool.
Damage and Risk Assessment A complete damage assessment in ERA Acute is based on quantification of impact (e. g. population/habitat loss) and the duration of the impact until recovery (pre-spill conditions). The Resource Damage Factor (RDF) is calculated as the integral of the extent of impact and duration of impact until recovery. For damage assessments, the probability for different outcomes is calculated and can inform on the likely damage which is typically the case for environmental impact assessments. The level of available or applicable temporal resolution of VEC data sets the frames for output resolution. For VEC data available as monthly distributions (e. g. seabirds, marine mammals), the impact and risk calculations can present impact levels and species at risk on a monthly basis.
2. 9). The foundation principle that ERA Acute uses continuous impact and damage algorithms, means it is well-suited to reflect even minor changes in spill scenarios or resources distribution as a change in impact and risk. In the ERA Acute software tool, a separate comparison module enables the user to compare oil spill trajectory, impact and risk data from different scenarios or calculations in a straightforward manner. This can be performed as compartment or resource impact and risk summaries but also plotted as impact maps. This is important in balancing the trade-offs by weighing and comparing the range of benefits and drawbacks associated with each response option (IPIECA-API-IOGP 2017). ISO 17776:2016 ISO 17776: (2016) )  Petroleum and natural gas industries-offshore production installationsmajor accident hazard management during the design of new installations.
Likelihood To perform Bayesian inference and get the probability distributions of the cosmological parameters, we used a Gaussian likelihood for a cosmology-independent covariance: log L(θ) = 1 2 (d -µ(θ)) T C -1 (d -µ(θ)), (6) where d is the data array, C is the covariance matrix of the observable, and µ is the expected theoretical prediction as a function of the cosmological parameters θ. In our case, the data array is the mean over the (simulated) realisations of all the separate observables, or, of their different combinations, for our fiducial model. Cosmological parameters are the ones for which simulations are available, namely {M ν , Ω m , 10 9 A s }, but the same statistics can of course be applied to other parameters if simulations allow. In order to predict the theoretical values of all observables given a new set of values for the cosmological parameters {M ν , Ω m , 10 9 A s }, we employed an interpolation with Gaussian process regression  (Rasmussen & Williams 2005)  using the scikit-learn python package.
MCMC simulations and posterior distributions We explored and constrained the parameter space with the emcee package, which is a python implementation of the affine invariant ensemble sampler for Markov chain Monte Carlo (MCMC) introduced by Foreman-Mackey et al. (  2013 ). We assumed a flat prior, a Gaussian likelihood function as defined in Sect. 6, and a model-independent covariance matrix as discussed in Sect. 3. 2. The walkers were initialised in a tiny Gaussian ball of radius 10 -3 around the fiducial cosmology [M ν , Ω m , 10 9 A s ] = [0. 1, 0. 3, 2. 1], and we estimated the posterior using 120 walkers. Our chains are stable against the length of the chain, and we verified their convergence by employing Gelman Rubin diagnos-   (Gelman & Rubin 1992). To plot the contours, we used the ChainConsumer python package  (Hinton 2016).
Acknowledgements. We thank  Jia Liu  and the  Columbia Lensing group  for making the MassiveNus (Liu et al. 2018) simulations available. Virginia Ajani acknowledges support by the  Centre National d'Etudes Spatiales  and the  project Initiative d'Excellence (IdEx) of Université de Paris  ( ANR-18-IDEX-0001 ). We thank the  Columbia Lensing group 4  for making their simulations available. The creation of these simulations is supported through grants  NSF   AST-1210877 ,  NSF   AST-140041 , and  NASA   ATP-80NSSC18K1093. We thank  New Mexico State University (USA)  and  Instituto de Astrofisica de Andalucia CSIC (Spain)  for hosting the Skies & Universes site for cosmological simulation product.