dataesr/scientific-paragraphs-categorization huggingface dataset:
Note that third and fourth inspections concern all females, three males kept for farm's own needs and FV-Polled1 itself. At the second or third inspection, the horn area of polled individuals was shaved to check for any small scurs or scabs. Furthermore, we collected phenotypic data for 15 heifers of FV-Polled1 (eight polled, seven horned) at two different dates. All the selected heifers were older than 13 months and kept on the same farm. The height at the withers, length from base of the neck to the base of the tail and length of labia majora were measured by measuring stick while the bodyweight of each animal was estimated with a weight measuring tape. All 15 heifers were www. nature. com/scientificreports/ macroscopically examined for any other apparent anomaly. The differences between polled and horned animals with respect to the phenotypic traits described as above were assessed using the following linear models: where, y i is the dependent variable related to body measurements of animal i, which is affected by fixed effects including the overall mean (µ), age at phenotyping (Age i in days) and horn status (HS i , polled = 1 and horned = 2).
Physical SNP marker positions were determined according to UMD3. 1 Bos taurus reference assembly  49. For quality control, genotype data were filtered for the following criteria: (i) GenCall score higher than 0. 95, (ii) missingness per marker less than 10%, (iii) missingness per individual less than 5%, (iv) minor allele frequency higher than 2. 5%. After quality control, 39,489 SNPs remained for analysis. Mapping of the de novo polled locus. The initial mapping of the novel polled phenotype was performed on genotyping data of 48 animals. This mapping population was designed before all 42 progenies of FV-Polled1 were born and thus include only 27 of these. Table  2  describes the mapping population. Mapping of the de novo polled locus was carried out using combined linkage disequilibrium and linkage analysis (cLDLA) corresponding to the method proposed by Meuwissen et al.
fa Sequenced_genome. bam | bcftools call -vmO z -o Sequenced_genome. vcf. gz). PromethION long-read sequencing. One µg of unsheared DNA was end-repaired and A-tailed with NEBnext UltraII End-repair and A-tailing module (New England Biolabs, Ipswich USA), purified with Ampu-reXP magnetic beads (Beckman Coulter, Brea USA), ligated to 1D sequencing adapters (LSK109 kit, Oxford Nanopore Technologies, Oxford, UK) and again purified with AmpureXP beads, according to the manufacturers' protocols. In the last Ampure cleanup step, washing with 70% ethanol was replaced with washing buffer supplied by Oxford Nanopore Technologies (LSK109 kit). Approximately 400 ng of adapted DNA library was loaded to a primed PromethION flowcell and sequenced for 65 h on a PromethION Alpha sequencer. Base calling of PromethION sequencing data was carried out using Albacore (v2.
Acknowledgements We are grateful to  Johannes Robeis  ( LfL Bayern ) who facilitated a farm for FV-Polled1 and visited the farm several times for phenotyping, as well as to his supervisor  Bernhard Luntz  ( LfL Bayern ) who permitted and financed his actions. Georg Nützel  ( AELF Bayern ) is acknowledged for farm visits and phenotyping. Dr. Georg Eller  ( Tierärztliche Klinik Dr. Eller ) is given thanks for providing samples and some here used observations. Finally, we thank  Horst Porzelt  the owner of the farm, who made this study possible in allowing all farm visits, he assisted in handling of the animals and contributed some of the here used observations. We are grateful for financial support of present study by funds from the  Tierzuchtforschung e. V. München , the  H. Wilhelm Schaumann Stiftung , the  Heinz Wüstenberg-Stiftung  and the  Karl Eibl-Stiftung.
Data availability The BovineSNP50 BeadChip genotyping data of the Fleckvieh animals are submitted in Figshare with https :// doi. org/10. 6084/m9. figsh are. 13013 885. The paired-end whole genome sequencing data of FV-Polled1 have been deposited in the European Nucleotide Archive (ENA) at EMBL-EBI under accession number PRJEB40548.
We present the MULTISEM systems submitted to SemEval 2020 Task 3: Graded Word Similarity in Context (GWSC). We experiment with injecting semantic knowledge into pre-trained BERT models through fine-tuning on lexical semantic tasks related to GWSC. We use existing semantically annotated datasets and propose to approximate similarity through automatically generated lexical substitutes in context. We participate in both GWSC subtasks and address two languages, English and Finnish. Our best English models occupy the third and fourth positions in the ranking for the two subtasks. Performance is lower for the Finnish models which are mid-ranked in the respective subtasks, highlighting the important role of data availability for fine-tuning.
Introduction The meaning of words is strongly tied to the context in which they occur: Different contexts might point to different senses or indicate subtler meaning nuances. SemEval 2020 Task 3 "Graded Word Similarity in Context" (GWSC)  (Armendariz and Purver, 2020)  explores the effect of context on meaning, and proposes to predict the similarity of word instances in a continuous, or graded, fashion. GWSC is based on the CoSimLex dataset  (Armendariz et al. , 2020)  and consists of two subtasks where models have to predict (1) the shift in meaning similarity for a pair of words (w a , w b ) occurring in different contexts, and (2) the similarity of two word instances in the same context. This is illustrated by sentences c 1 and c 2 , two contexts where dinner and breakfast co-occur. c 1 (.
) After Mickey rings the dinner bell, Goofy foolishly leaves the driver's seat for breakfast. c 2 Residence Inns typically feature a complimentary small hot breakfast in the morning and a free light dinner or snack reception on weekday evenings (. ) A change in meaning similarity occurs between the highlighted words in the two sentences: They are less similar in context c 1 where dinner is part of a noun compound (dinner bell), than in context c 2 where they describe different kinds of meals. The shift in meaning is reflected in the gold similarity scores assigned to these instance pairs in the GWSC dataset (4. 39 vs. 5. 35). We build models for these two subtasks by fine-tuning BERT on existing lexical similarity datasets. Additionally, we propose to approximate similarity of words in context through automatically generated lexical substitutes.
Datasets We fine-tune pre-trained BERT models on semantic tasks that are related to GWSC. We specifically select tasks that address the similarity of word meaning in context, and use the corresponding datasets to make BERT more sensitive to this specific aspect of meaning. Table  1  contains annotated instances from each dataset used in our experiments. Usim The Usim dataset contains 10 sentences for each of 56 words of different parts of speech, manually annotated with pairwise usage similarity scores  (Erk et al. , 2009; Erk et al. , 2013). 2  As in GWSC, similarity scores are graded and range from 1 (completely different) to 5 (same meaning). The Usim sentences come from the SemEval 2007 Lexical Substitution task dataset  (McCarthy and Navigli, 2007). 3 To binarize the usage similarity scores and use them for fine-tuning, we consider sentence pairs annotated with low similarity scores (score < 2) as instances denoting a different meaning (F), and highly similar sentence pairs (score > 4) as instances of the same sense (T).
Concepts in Context (CoInCo) The CoInCo corpus  (Kremer et al. , 2014)  contains manually selected substitutes for all content words in a sentence. Substitute overlap between different word instances reflects their semantic similarity: instance pairs with similar meaning share a higher number of substitutes. We binarize the data as in Garí Soler et al. (  2019 ) by assigning instance pairs to a class describing the same (T) or different (F) meaning depending on their shared substitutes. The data sample used by Garí Soler et al. contains instances with at least four substitutes: T pairs involve instances that have at least 75% of substitutes in common, and F pairs correspond to instances with no substitute overlap. We gather additional data from CoInCo by relaxing the class inclusion constraints.
Specifically, we retain all instances regardless of the number of available substitutes. We consider as T examples instance pairs that have at least 50% of substitutes in common, and as F examples pairs that share at most one substitute. We retain up to 500 instance pairs per CoInCo lemma, when available. We balance the two classes (T and F) and merge the obtained instances with Garí Soler et al. (  2019 )'s dataset (5,023 pairs) removing the duplicates. In total, we have 22,226 CoInCo instance pairs for fine-tuning. We use these instances in combination with the Usim data. 1998) , VerbNet  (Schuler, 2006)  and Wiktionary examples, and were automatically annotated based on information provided in these resources. We use the training set (5,428 sentence pairs) with its labels (T or F) as data for fine-tuning.
ukWaC-subs The GWSC task addresses pairs of different words that can have similar meanings in some contexts and not in others (e. g. , room and cell). Given that no training data is available, we automatically create one more dataset for fine-tuning called ukWaC-subs, which approximates this task. ukWaC-subs contains pairs of sentences (p 1 , p 2 ) that differ in one word only. We create data by substituting a word w in p 1 by either (a) a correct substitute; (b) a word that is a good synonym of w and could have been a correct substitute in another context but not in this one; or (c) a random word of the same part of speech as w. This is illustrated by the three ukWaC-subs sentences in Table  1. With (a), we expect BERT to learn that clear is being used in its unambiguous sense in this context.
In (b), we tell BERT that despite the (out-of-context) similarity between present and moment, the latter is not adequate in this context. With (c), we want BERT to learn to distinguish date from a completely unrelated word (heritage). We use this data for a 3-way classification task. We create this dataset by gathering sentences from the ukWaC corpus  (Baroni et al. , 2009)  and automatically annotating them with lexical substitutes. We identify the content words in a sentence and use as their candidate substitutes their paraphrases in the Paraphrase Database (PPDB) lexical XXL package  (Ganitkevitch et al. , 2013; Pavlick et al. , 2015). 4 The PPDB resource was automatically constructed by a bilingual pivoting method. Every paraphrase pair has a PPDB 2. 0 score indicating its quality. We only consider as candidates for substitution pairs with a score above 2.
We create 100,000 instances that we use to fine-tune BERT. OpusParcus  Shi et al. (2019)  show that retrofitting ELMo with paraphrases improves its performance on lexical semantic tasks. We follow a similar approach and use paraphrases to fine-tune BERT before applying it to GWSC. We use paraphrases from the Open Subtitles Paraphrase Corpus (Opusparcus)  (Creutz, 2018). We use this corpus instead of the Microsoft Research Paraphrase Corpus  (Dolan et al. , 2004)  used by  Shi et al. (2019) , because it contains paraphrase pairs for six European languages, including English and Finnish which are addressed in GWSC. Paraphrase pairs in Opusparcus were extracted from movies and TV shows subtitles, and are ranked by quality. We use paraphrases from the Opusparcus training set with a quality score higher than 15, 5  and create our own training instances following the procedure of  Shi et al.
Models We use these five datasets to fine-tune pre-trained BERT models for English and Finnish. All tasks require comparing the meaning of word instances in two different sentences. We form an input sequence (sentence pair) for BERT by joining the two sentences together with the separator token  ([SEP] ) in between. Since the task is at the word level, we do not build our classifier on top of the [CLS] token which is an aggregation of the whole input sequence. Instead, our classifier receives as input the BERT representations of the target word instances at the last layer. BERT uses wordpiece tokenization  (Wu et al. , 2016)  which means that a target word may be split into several tokens. For words that have been split, we average the representations of each wordpiece. We use two kinds of heads for fine-tuning.
• Classification head: The representations of the two target tokens are concatenated and fed to a linear classifier which outputs probabilities for each class. We use a cross entropy loss for training. We call this head CLASSIF. • Cosine Distance head: We apply the Cosine Embedding Loss (PyTorch  (Paszke et al. , 2019) ) to the representations of the two target tokens at the last layer. This loss increases the cosine distance of two tokens if they do not have the same meaning, and decreases it otherwise. We refer to this head as COSDIST. Note that the ukWaC-subs dataset is compatible with the CLASSIF head only because it has three classes. To predict the similarity of two target tokens in the GWSC task, we extract their representations from the different layers of a fine-tuned model. We use cosine similarity (cossim) as our similarity metric.
Experimental Setup We participate in Subtasks 1 and 2 for English and Finnish. 6 For English, we fine-tune the bert-base-uncased model. For Finnish, we use the uncased Finnish model (finnish) (Virtanen et al. , 2019) 7  and the uncased Multilingual BERT-base model (multilingual). 8 For faster fine-tuning, we set the maximum length to 128 wordpiece and omit examples where a target word occurs after this position. As a development set for English, we use the officially released GWSC trial data (10 sentence pairs) and an earlier release of trial data (8 sentence pairs), both distinct from the test set. We use these data to select the best models and hyperparameters for our official submissions to GWSC. The English test set consists of 340 context pairs for Subtask 1 and 680 unique contexts for Subtask 2.
We finetune bert-base-uncased separately on each of our English datasets experimenting with the two classification heads {CLASSIF, COSDIST} and with different learning rates {5e-5, 1e-6, 1e-7} for up to 15 epochs. These hyperparameters, along with the layer the word representations are extracted from, are set on the GWSC trial data. Our submitted models were fine-tuned on WiC, Opusparcus and CoInCo-Usim with a learning rate of 5e-5 and 0. 1 dropout for 4, 3 and 2 epochs, respectively. The ukWaC-subs model was fine-tuned for 11 epochs with a learning rate of 1e-6 and 0. 2 dropout. Dropout was determined based on results on 2,000 held-out ukWaC-subs instances. Since no trial dataset was released for Finnish, we fixed the hyperparameters for our models to those that worked best for the English Opusparcus data.
Conclusion We have participated in the SemEval task Graded Word Similarity in Context for English and Finnish, with models integrating different notions of word similarity. We have specifically investigated the effect of fine-tuning pre-trained BERT models on existing datasets that address word meaning similarity in context. Furthermore, we have proposed a new fine-tuning task where in-context lexical similarity is approximated through automatic substitute annotations. Our English models are ranked at the third and fourth position in the two GWSC subtasks, and outperform a BERT-based baseline without fine-tuning. This demonstrates the benefit of fine-tuning BERT on a task that is closely related to the end task, even when the data used for fine-tuning are automatically obtained. Due to the scarcity of resources for Finnish, we could only fine-tune models with paraphrases.
For neuroscientists, the message was unambigu-ous. b (F) Need a present for someone with a unique name?Need a moment for someone with a unique name?c (F') Overdue tasks display on the due date. Overdue tasks display on the due heritage. OpusparcusT I love you so much I love you to the moon and back. F yes, Mary, I would love to dance. Why do I love him? Table 1: Examples of instance pairs from each dataset used for fine-tuning. Word-in-Context (WiC) The WiC dataset contains pairs of word instances in context with the same or a different meaning (Pilehvar and Camacho-Collados, 2019). Sentences come from WordNet (Fellbaum,.
The aim of this paper is to assess the potential use of data recorded by the Global Navigation Satellite System Reflectometry (GNSS-R) Cyclone Global Navigation Satellite System (CYGNSS) constellation to characterize desert surface roughness. The study is applied over the Sahara, the largest non-polar desert in the world. This is based on a spatio-temporal analysis of variations in Cyclone Global Navigation Satellite System (CYGNSS) data, expressed as changes in reflectivity (Γ). In general, the reflectivity of each type of land surface (reliefs, dunes, etc. ) encountered at the studied site is found to have a high temporal stability. A grid of CYGNSS Γ measurements has been developed, at the relatively fine resolution of 0. 03 • × 0. 03 • , and the resulting map of average reflectivity, computed over a 2.
5-year period, illustrates the potential of CYGNSS data for the characterization of the main types of desert land surface (dunes, reliefs, etc. ). A discussion of the relationship between aerodynamic or geometric roughness and CYGNSS reflectivity is proposed. A high correlation is observed between these roughness parameters and reflectivity. The behaviors of the GNSS-R reflectivity and the Advanced Land Observing Satellite-2 (ALOS-2) Synthetic Aperture Radar (SAR) backscattering coefficient are compared and found to be strongly correlated. An aerodynamic roughness (Z 0 ) map of the Sahara is proposed, using four distinct classes of terrain roughness.
Various studies have been proposed to estimate land surface properties, such as soil moisture and vegetation biomass  [34] [35] [36] [37] [38]. The aim of the present study is to analyze, for the first time, the potential of CYGNSS GNSS-R data for desert surface applications. This paper is organized as follows: Section 2 provides a general description of the studied site and the datasets considered in this study; Section 3 describes the methodology used to interpret and map the CYGNSS data recorded over the Sahara; Section 4 focuses on the results, and discusses the relationship between reflectivity and surface roughness (aerodynamic or geometric); finally, our conclusions are provided in Section 5.
Satellite Data a) CYGNSS data The main dataset used for this analysis is the Level 1 (L1) CYGNSS 0 data, version 2. 1, obtainable from the Physical Oceanography Distributed Active Archive Center (PODAAC) (https://podaac. jpl. nasa. gov). The L1 data are stored as daily Network Common Data Form (NETCDF) files, in which each day is represented by eight files (one for each CYGNSS platform), with each of these containing a large number of useful variables that can be used for the subsequent processing steps. The main pieces of information are the Delay/Doppler Maps (DDMs) of the analogue scattered power. For each data point, the specular point properties (i. e. , the DDM sample corresponding to the point on the surface minimum electromagnetic path length) are taken into account, together with several geometrical parameters relevant to each acquisition (e.
g. , specular point incidence angle, transmitter and receiver ranges, etc. ), and metadata related to the quality of the incoming and reflected GPS signals (e. g. , direct Signal to Noise Ratio (SNR), receiver gain, SNR for each DDM etc. ). In this research, almost 2. 5 years of CYGNSS acquisitions, recorded between March 2017 and July 2019, have been processed and analyzed. In order to filter the data correctly, bitwise quality flags are provided to select the best data, and in the present study, all available quality flags were considered and set to 1  [41, 42]. The theoretical footprint of a reflected GNSS signal is ~0. 5 × 0. 5 km, for the case of a smooth surface and a receiver at the altitude of CYGNSS. The receivers integrate signals over 1 s. During this period of time, the receiver travels ~7 km.
Satellite Data (a) CYGNSS data The main dataset used for this analysis is the Level 1 (L1) CYGNSS 0 data, version 2. 1, obtainable from the Physical Oceanography Distributed Active Archive Center (PODAAC) (https://podaac. jpl. nasa. gov). The L1 data are stored as daily Network Common Data Form (NETCDF) files, in which each day is represented by eight files (one for each CYGNSS platform), with each of these containing a large number of useful variables that can be used for the subsequent processing steps. The main pieces of information are the Delay/Doppler Maps (DDMs) of the analogue scattered power. For each data point, the specular point properties (i. e. , the DDM sample corresponding to the point on the surface minimum electromagnetic path length) are taken into account, together with several geometrical parameters relevant to each acquisition (e.
The noise floor is computed as the mean value of the DDM subset, when the signal is absent (located above the characteristic horseshoe shape of the DDMs) as suggested, for example, in  [35]. The data are collected in daily data frames, taking into account various empirical filters used to remove noisy data, which could otherwise be expected to degrade the quality of the imagery and subsequent analysis. The filtering criteria considered for this purpose were the following: (1) CYGNSS reflectivity values should range between -35 dB and -5 dB; (2) Incidence angles should range between 0 • and 30 • ; (3) DDM signal-to-noise ratio (SNR) should be greater than 3 dB; (4) G rx (receiver antenna gain towards SP) should be greater than 5 dB; (5) Data corresponding to terrain at altitudes greater than 650 m above the WGS84 ellipsoid model of the Earth should be removed.
(b) ALOS-2 data The open dataset provided by the ALOS-2 mission, an advanced L-band platform developed by Japan Aerospace Exploration Agency (JAXA), is considered. The data are recorded at a global resolution of 25 m, created by mosaicking SAR images of backscattering coefficients, which were measured by the Phased Array type L-band Synthetic Aperture Radar 2 (PALSAR-2), freely available at the JAXA Earth Observation Research Center (EORC/JAXA) (https://www. eorc. jaxa. jp/ALOS/en/ palsar_fnf/data/index. htm). Each file contains pre-processed information related to the local incidence angle, acquisition date, and the backscattering coefficient for the horizontal-horizontal (HH) and horizontal-vertical (HV) polarizations. The pre-processing includes ortho-rectification (correction of geometric distortion) and slope correction (topographic effects on image intensity).
In the present paper, HH backscattering maps from the year 2017 are used, whereas the HV data are not analyzed due to their greater sensitivity to volume rather than surface scattering. As described in the PALSAR-2 handbook  [46] , the backscattering maps are not calibrated, and the results are provided in the form of digital counts. In practice, a calibration coefficient is needed to transform the product into a more convenient decimal format, according to the following expression: σ o | dB = 10 log 10 DN 2 + CF (2) where σ 0 is the output backscattering coefficient in decibels, DN is the input digital number contained in the dataset, and CF is the calibration factor, which is set to -83 dB  [47].
Ground Measurements of Aerodynamic Roughness In this study, we made use of ground measurements of aerodynamic roughness, measured over the Sahara desert, provided in the study of  Marticorena et al. (1997)   [5]. As listed in Table  1 , 23 areas are considered, corresponding to a wide range of surface conditions (dunes, smooth surfaces, reliefs etc. ). The same database was also used in the past to develop aerodynamic roughness maps, based on data recorded by the ERS scatterometer  [4].