dataesr/scientific-paragraphs-categorization huggingface dataset:
The uncertainties are estimated by comparing data and simulation in the pile-up-and hard-scatter-enriched regions, respectively. The hard-scatter efficiency is found to be underestimated in simulation. The level of disagreement is observed to be larger at low jet p T and high |η| and can be as large as about 3%. The efficiencies evaluated in this paper are used to define a calibration procedure accounting for this discrepancy. The uncertainties associated with the calibration and resolution of the jets used to compute fJVT are estimated in ATLAS analyses by recomputing fJVT for each variation reflecting a systematic uncertainty. 6 Pile-up jet tagging with shape and topological information The fJVT and γ discriminants correspond to a twofold strategy for pile-up rejection targeting QCD and stochastic pile-up jets, respectively.
In events with 35 pile-up interactions, typical conditions for the LHC operations in the near future, 37%, 48%, and 51% of forward pile-up jets are rejected using, respectively, topological information, shape information, and their combination, for the same 85% hard-scatter efficiency. A procedure is defined and used to measure the efficiency of identifying hard-scatter jets in 3. 2 fb -1 of pp collisions at √ s = 13 TeV collected in 2015. The efficiencies are measured in data and estimated in simulation as a function of the jet kinematics. Discrepancies of up to approximately 3% are observed, mainly due to the modelling of pile-up events. The impact of forward pile-up rejection algorithms presented here is estimated in a simplified study of Higgs boson production through vector-boson fusion and decaying into a ττ pair; the signal purity for the baseline selection under consideration, where Z+jets production is the dominant background, can be enhanced by a factor of about four for events with 35 pile-up interactions.
Alternatively or additionally, the lower percentage of peptides derived from TMHs might be caused by a lower detection efficiency, as hydrophobic peptides are often underrepresented in mass spectrometry due to their  poor solubilization and ionization characteristics  (25, 26). In any case, this survey of peptide elution studies  (24)  shows that MHC-I presents epitopes derived from TMHs. In fact, many known MHC-I-binding epitopes locate within transmembrane helices. For HLA-A02, which is the most prevalent HLA-A haplotype in the Caucasian population  (27) , more than 10% of the epitopes in the Immune Epitope Database (28) correspond to peptide fragments overlapping with TMHs. These include many well-studied and clinically important epitopes that were found in in vivo studies. For example, multiple HLA-A02-binding epitopes are derived from the TMH of the viral coat protein gp41 from the human immunodeficiency virus  (29).
In Silico predictions Transmembrane helices for all human proteins were predicted using the TMHMM Server 2. 0 1  (23). HLA-binding 9-mers were predicted using a custom R implementation 2 of the SMMPMBEC 1 http://www. cbs. dtu. dk/services/TMHMM/. 2 https://github. com/jtextor/epitope-prediction. method  (46). For each HLA super type, IC50 values for all 9-mers from the human proteome were predicted. The 2% peptides with the lowest IC50 values were defined as binders.
This suggested that the a°module is likely the major contributor for the oxidase activity measured with the intact protein and that there may be a partial compensation by the second domain in the absence of the catalytic cysteines of the a°module. Then, the reductase activity of the respective PDI-M domains was tested by assessing their capacity to reduce insulin and to activate NADP-MDH in the presence of DTT. In the insulin reduction assay, PDI-M(a) was more efficient than PDI-M(a°) (Fig  6A ). In this case, the fact that the activity of both domains was decreased compared to the activity of PDI-M suggests that both domains contribute to the reductase activity of PDI-M although with different efficiencies. In the NADP-MDH activation assay, we observed that PDI-M was the only PDI representative able to activate the enzyme since PDI-A (data not shown) and PDI-L1a were unable to do that (Fig  6B ).
This was quite surprising taking into account the unfavourable thermodynamic barrier, NADP-MDH possesses disulfides with very negative redox potential, around -300 mV at pH 7. 0  [36]. However, the observed activation was less efficient compared to the one observed at identical concentrations with Trx h1 used as a reference. The two PDI-M domains presented different reactivities toward oxidized NADP-MDH, PDI-M(a) was even slightly more active than PDI-M whereas PDI-M(a°) was less active (Fig  6B ). Altogether, these data indicated that the a module is a more efficient reductant than the a°module. The properties of PDI-M domains have been further examined using the DTNB assay and AtNTRB as an electron donor. The PDI-M(a°) domain was as efficient as PDI-M whereas PDI-M(a) was only very poorly able to promote DTNB reduction (Fig  7A ).
006G010400; PpPDI-L1, Phpat. 015G023600; PpPDI-M, Phpat. 004G043700); Pt, Populus trichocarpa (PtPDI-A, Potri. 009G004500; PtPDI-L1a, Potri. 002G082100; PtPDI-M, Potri. 014G160000) and Zm, Zea mays (ZmPDI-A, GRMZM2G073628; ZmPDI-L1a, GRMZM2G091481; ZmPDI-M, GRMZM2G389173). The Trx modules were delimited according to Pfam only database (http:// pfam. sanger. ac. uk/) and the alignment was built using ClustalW algorithm at the NPSA web portal (http://npsa-pbil. ibcp. fr). Output of this alignment was made with the ESPript web portal (http://espript. ibcp. fr/ESPript/cgi-bin/ESPript. cgi). Amino acids strictly conserved appear in black whereas partially conserved amino acids are indicated in light grey. Cysteines of the active site signature are indicated with stars, The E, K, R residues also represented in the panel A and possibly involved in the modulation of the cysteine pKa are indicated by triangles.
To compare particular sections of sampled regions, all pictures were taken with the same magnification (x100). Sections were examined with a ZEISS Axioskop and pictured with ZEISS AxioCam ERc 5s camera using ZEISS AxioVision Release 4. 8. 2 software (ZEISS, Goettingen, Germany). Collection and analysis of scent samples. We performed scent collection on six wild brown bears captured and immobilized in the northern Carpathians (S Poland). Bears were captured in a modified culvert trap or Aldrich traps and immobilized following the biomedical protocols by Arnemo et al. 44  in 2014-2016. All capture and handling procedures were approved by the Polish authorities and the Ethical Committee (see ethics statement). The GC/MC analyses were carried out by gas chromatography laboratory at the Wrocław Research Centre EIT+, Wrocław, Poland.
Acknowledgements This study was funded by the  Norway Grants  under the  Polish-Norwegian Research Program  operated by the  National Centre for Research and Development (GLOBE ,  POL-NOR/198352/85/2013 ). The research visits of the teams cooperating in this study were funded by the  National Science Center in Poland  (project  DEC-2013/08/M/ NZ9/00469 ). Histological work and imaging were funded through internal grant of the  University of Wrocław  ( 1018/S/IBS/10/2016 ). We thank  E. Piątkowska  from the mass spectrometry lab at the  Wrocław Research Centre EIT+  for advice on extraction methods, and  K. Chrząścik  for drawing the graphical abstract. D. Huber ,  A. Krzeptowski-Sabała ,  M. Klimecki ,  R. Mateja ,  P. Łukaszczyk ,  S. Zięba ,  J. Ficek ,  K. Zabiega  and  M. Pasiniewicz  assisted with fieldwork.
The WebNLG challenge consists in mapping sets of RDF triples to text. It provides a common benchmark on which to train, evaluate and compare "microplanners", i. e. generation systems that verbalise a given content by making a range of complex interacting choices including referring expression generation, aggregation, lexicalisation, surface realisation and sentence segmentation. In this paper, we introduce the microplanning task, describe data preparation, introduce our evaluation methodology, analyse participant results and provide a brief description of the participating systems.
Introduction Previous Natural Language Generation (NLG) challenges have focused on surface realisation  (Banik et al. , 2013; Belz et al. , 2011) , referring expression generation  (Belz and Gatt, 2007; Gatt et al. , 2008; Gatt et al. , 2009; Belz et al. , 2008; Belz et al. , 2009; Belz et al. , 2010)  and content selection  (Bouayad-Agha et al. , 2013). In contrast, the WebNLG challenge focuses on microplanning, that subtask of NLG which consists in mapping a given content to a text verbalising this content. Microplanning is a complex choice problem involving several subtasks referred to in the literature as referring expression generation, aggregation, lexicalisation, surface realisation and sentence segmentation. For instance, given the WebNLG data unit shown in (1a), generating the text in (1b) involves choosing to lexicalise the JOHN E BLAHA entity only once (referring expression generation), lexicalising the OCCUPATION property as the phrase worked as (lexicalisation), using PP coordination to avoid repeating the word born (aggregation) and verbalising the three triples by a single complex sentence including an apposition, a PP coordination and a transitive verb construction (sentence segmentation and surface realisation).
Data As illustrated by the above example, the WebNLG dataset was designed to exercise the ability of NLG systems to handle the whole range of microplanning operations and their interactions. It was created using a content selection procedure specifically designed to enhance data and text variety  (Perez-Beltrachini et al. , 2016). In  (Gardent et al. , 2017) , we compared a dataset created using the WebNLG process with existing benchmarks in particular,  (Wen et al. , 2016) 's dataset (RNNLG) which was produced using a similar process. In what follows, we give various statistics about the WebNLG dataset using the RNNLG dataset as a reference point. Size. The WebNLG dataset consists of  25,298 (data,text)  pairs and 9,674 distinct data units. The data units are sets of RDF triples extracted from DBpedia and the texts are sequences of one or more sentences verbalising these data units.
Lexicalisation. As illustrated by the examples in (2), different properties can induce different lexical forms (a property might be lexicalised as a verb, a relational noun, a preposition or an adjective). Therefore, the larger the number of properties, the more likely the data is to allow for a wider range of lexicalisation patterns. ( 2) X TITLE Y ⇒ X served as Y Verb X NATIONALITY Y ⇒ X's nationality is Y Relational noun X COUNTRY Y ⇒ X is in Y Preposition X NATIONALITY USA ⇒ X is American Adjective To promote diverse lexicalisation patterns, we extracted data from 15 DBpedia categories (Astronaut, University, Monument, Building, ComicsCharacter, Food, Airport, SportsTeam, WrittenWork, Athlete, Artist, City, MeanOfTransportation, CelestialBody, Politician) resulting in a set of 373 distinct RDF properties (more than three times the number of properties contained in the RNNLG dataset).
The corrected type token ratio (CTTR 1 ) and the number of word types is roughly twice as large in the WebNLG dataset than in RNNLG. Surface Realisation. To increase syntactic variety, we use a content selection procedure which extracts data units of various shapes. The intuition is that different input shapes may induce distinct linguistic constructions. This is illustrated in Figure  1. Typically, while triples sharing a subject (SIBLING configuration) are likely to induce a VP or a sentence coordination, a CHAIN configuration (where the object of one triple is the subject of the other) will more naturally give rise to object relative clauses or participials. Another factor impacting syntactic variation is the set of properties (input patterns) cooccurring in a given input. This is illustrated by the examples in (3) where two inputs of the same length (3 triples hence 3 properties) result in text with different syntax.
That is, a larger number of input patterns is more likely to induce texts with greater syntactic variety. By extracting data units from a large number of distinct domains (DBpedia categories), we seeked to produce a large number of distinct input patterns. 1 Following (Perez-Beltrachini and Gardent, 2017), we use  (Lu, 2008) 's system to compute the CTTR  (Carroll, 1964). (3) a. As shown in Table  1 , the WebNLG dataset contains twice as many distinct input patterns and ten times more input shapes than the RNNLG dataset. It is also less redundant with a ratio between number of inputs and number of input patterns of 2. 34 against 10. 31 for RNNLG. Aggregation, Sentence Segmentation and Referring Expression Generation. Finally, the need for aggregation, sentence segmentation and referring expression generation mainly arises when texts contain more than one sentence.
As Table  1  shows, although data units are overall smaller in the WebNLG dataset than in RNNLG, the WebNLG dataset has a higher number of texts containing more than one sentence and contains texts of longer length. Netherlands (UTILBURG), University of Information Technology, VNU-HCM, Vietnam (UIT-VNU-HCM) and Universitat Pompeu Fabra, Barcelona, Spain (UPF-FORGE). Each team submitted outputs from a single system except UTILBURG who submitted outputs from three different systems. As a result, there were nine systems in total: eight participating systems and our baseline (BASELINE) system. These can be grouped into three categories: pipeline systems, statistical machine translation (SMT) and neural machine translation (NMT) systems. Table  2  shows the system categorisations.
Participating Systems Pipeline Systems. Three submissions used a template or grammar-based pipeline framework with some NLG module: UTILBURG-PIPELINE, UIT-VNU-HCM and UPF-FORGE. The first two systems, UTILBURG-PIPELINE and UIT-VNU-HCM, extracted rules or templates from the training data for surface realisation, whereas the third system, UPF-FORGE, used the FORGe grammar  (Mille et al. , 2017). UTILBURG-PIPELINE extracted rules mapping a triple (or a triple set) to a text observed in the train- ing data; both the triple and the associated text were delexicalised. Given an RDF triple set to generate from, UTILBURG-PIPELINE first ordered triples to maintain discourse order. Extracted rules were then applied to generate a delexicalised text. Missing entities were added using a referring expression generation module  (Castro Ferreira et al.
SMT Systems. UTILBURG-SMT was the only system which used the statistical machine translation framework. It was trained on the WebNLG dataset using the Moses toolkit  (Koehn et al. , 2007). The dataset was pre-processed whereby each entity in the input and each corresponding referring expression in the output were delexicalised and annotated with the entity Wikipedia ID. The alignments from the training set were obtained using MGIZA and model weights were tuned using 60batch MIRA with BLEU as the evaluation metric. Similar to UTILBURG-PIPELINE, the system used a 6-gram language model trained on the Gigaword corpus using KenLM.
An n-gram search is used to assure the most accurate target sequence delexicalisation. They use a standard encoder-decoder with attention model. UTILBURG-NMT is based on the Edinburgh Neural Machine Translation submission for the 2016 machine translation shared task (WMT 2016). The target sequences are the delexicalised texts (cf. UTILBURG-PIPELINE) and the input sequences are the linearisation of the delexicalised input set of triples. The REG module from their pipeline system is used to post-process the decoder outputs. The PKUWRITER system relies upon two extra mechanisms, namely a ranking module and an extra Reinforcement Learning (RL) training objective. It uses an ensemble of attention-based encoderdecoder models based on the TensorFlow seq2seq API in addition to the baseline (7 models in total).
They propose an output ranking module to choose the best verbalisation among those outputs by the generation models. The ranker is trained on supervised data generated automatically. Input triple sets are paired with verbalisations produced by each of the generation models. Then, each pair is associated with a quality score, i. e. the BLEU score of the verbalisation and the reference. Word and sentence level features are extracted to train the ranker. The generation models and ranker are trained on different dataset partitions. The RL objective aims at encouraging the generator to include triple subjects in the produced verbalisation. Finally, PKUWRITER uses a set of hand-crafted rules to handle input cases where the model fails.
Automatic Evaluation Three automatic metrics were used to evaluate the participating systems: • BLEU-4 3  (Papineni et al. , 2002). BLEU scores were computed using up to three references. • METEOR (v1. 5) 4 (Denkowski and Lavie, 2014); • TER 5  (Snover et al. , 2006). For statistical significance testing, we followed the bootstrapping algorithm described in  (Koehn and Monz, 2006). To assess the ability of the participating system to generalise from out-of-domain data, the test dataset consists of two sets of roughly equal size: a test set containing inputs created for entities belonging to DBpedia categories that were seen in the training data (Astronaut, University, Monument, Building, ComicsCharacter, Food, Airport, SportsTeam, City, and WrittenWork), and a test set containing inputs extracted for entities belonging to 5 unseen categories (Athlete, Artist, MeanOfTransportation, Ce-lestialBody, Politician).
We call the first type of data seen categories, the second, unseen categories. Correspondingly, we report results for 3 datasets: the seen category dataset, the unseen category dataset and the total test set including both data from seen and from unseen categories. While in the seen test data (first column) almost all triple elements are present in the training and development sets, in the unseen test data (second column) the vast majority of subjects, objects, and, more importantly, properties (which need to be verbalised) has not been seen in the training and development data. Participants were requested to submit tokenised and lowercased texts. To ensure consistency between submissions, we pre-process the submitted results one more time to double-check that those requirements were fullfilled.
Results We briefly discuss the automatic scores distinguishing between results on the whole dataset, on data extracted from previously unseen categories and on data extracted from seen categories. Global Scores. Table  5  shows the global results that is, the results for the whole test set. Horizontal lines group together systems for which the difference in scores is not statistically significant. The names of the teams are coloured according to the system type they explored: neural-based systems are in red, pipeline systems in blue, and SMT systems in light grey. Most systems (6 out of 8) outperform the baseline, four of them obtaining scores well above it. In terms of BLEU and TER scores, the four first systems include systems of each type (neural, SMT-based and pipelines). While BLEU and TER yield almost identical rankings, METEOR does not, suggesting that the systems handle synonyms and morphological variation differently.
In particular, the fact that UPF-FORGE ranks first under the METEOR score suggests that it often generates text that differs from the references because of synonymic or morphological variation. Scores on Seen Categories. For data extracted from DBpedia categories that were seen in the training data, machine learning based systems (neural and SMT) mostly outperform more rule-based systems. Notably, in terms of BLEU and TER scores, the three pipeline systems are at the end of the rating. Again though, the METEOR scores show a much higher ranking (3rd rather than 6th) for the UPF-FORGE system. Scores on Unseen Categories. On unseen categories, the UPF-FORGE systems ranks first as the system could quickly be adapted to handle properties that had not been seen in the training data. The  ranking of the other systems is more or less unchanged with the exception of the ADAPTCENTRE system.
Conclusion The WebNLG challenge was novel in that it was the first challenge to provide a benchmark on which to evaluate and compare microplanners. Despite a tight schedule (we released the training data in April for a submission in August), it generated a high level of interest among the NLG community: 62 groups from 18 countries 6 downloaded the data, 6 groups submitted 8 systems and 3 groups developed a system but did not submit. The training data for the WebNLG 2017 challenge is available on the WebNLG website 7 and evaluation on the test data can be run by the organisers on demand. A larger dataset consisting of 40,049 (data, text) pairs, 15,095 distinct data input and 15 DBpedia categories is also available. Both datasets are under the Creative Commons license "CC Attribution-Noncommercial-Share Alike 4.
0 International license". We hope that these resources will enable a long and fruitful strand of research on microplanning. The usefulness of the WebNLG dataset reaches far beyond the WebNLG challenge. It can be used for instance to train a semantic parser which would convert a sentence into a set of RDF triples. It can also be used to derive new datasets for related tasks. Thus in  (Narayan et al. , 2017) , we show how to derive from the WebNLG dataset a dataset for sentence simplification which we call the Splitand-Rephrase dataset. In this dataset, each pair consists of (i) a single, complex sentence and its meaning representation in terms of RDF triples and (ii) a sequence of at least two sentences and their corresponding RDF triples. In other words, the Splitand-Rephrase dataset associates a complex sentence with a sequence of at least two sentences whose meaning is the same as that of the complex sen-6 Australia, Canada, China, Croatia, France, Germany, India, Iran, Ireland, Italy, Netherlands, Norway, Poland, Spain, Tunisia, UK, USA, Vietnam 7 http://talc1.
loria. fr/webnlg/stories/ challenge. html tence. As explained in  (Narayan et al. , 2017) , this dataset was created using the meaning representations (sets of RDF triples) as pivot. The Splitand-Rephrase dataset consists of 1,100,166 pairs of the form (M C , T C ), {(M 1 , T 1 ). (M n , T n )} where T C is a complex sentence and T 1. T n is a sequence of texts with semantics M 1 ,. M n expressing the same content M C as T C. Figure  2  shows an example pair. It was used to train four neural systems, and the associated meaning representations were shown to improve performance. In the future, we are planning to build a multilingual resource in which the English text present in the WebNLG dataset will be translated into French, Russian and Maltese. In this way, morphological variation can be explored which is an interesting avenue of research in particular for neural systems which have a limited ability to handle unseen input: how well will these systems be able to handle the generation of morphologically rich languages? The analysis of the participants' results presented in this paper will be complemented in an arXiv report by the results of a human-based evaluation.
In order to maintain and measure attention, participants were told to press a response button as fast as possible if a green square replaced the face displayed (the green squares appeared 500 ms after the stimulus onset). Thirty of those 'active' trials were randomly interleaved with the passive displays. To avoid biases, the 6 stimuli of interest were seen the same number of times (i. e. , 5) in the active condition. The association between the 4 neutral faces and the social ranks were counterbalanced across subjects using a Latin square procedure, so that activities related to rank could not be explained by facial traits. The neutral faces of the 4 male players were taken from standardized database (the NimStim Set of Facial Expressions, www. macbrain. org/resources. htm). Figure  1  only displays the face of these players and not any participant's face, for confidentiality reasons.