dataesr/scientific-paragraphs-categorization huggingface dataset:
Contributions In our work we address the challenges described above-country-scale data with high temporal frequency, sparse annotations across varying climatic regions, imprecise segmentation produced by standard deep CNN approaches-by making these contributions: 1. a framework of patch generation, data augmentation and patch distribution in a computing cluster that alleviates the very high class imbalance in the dataset; 2. a novel adaptation of the U-Net model for dealing with sparse data, while producing detailed and fine-grained class predictions; 3. we show results in an operational setting on the country scale for multi-temporal multispectral production-quality data obtained from the Sentinel-2 constellation.
Remote Sensing Imagery Eleven Sentinel-2 tiles (100 km × 100 km) covering different areas of metropolitan France were selected. They are shown in Figure  1. For each tile, all available Sentinel-2 through the Theia Land Data Centre (http://www. theia-land. fr/en/presentation/products) for the year 2016 were used. The Sentinel-2 products are orthorectified prior to their release by ESA (Level-1C) and then, Theia processing chains based on the algorithms described in  [27]  perform cloud (and cloud shadow) screening and atmospheric corrections to produce the Level-2A imagery used in this work. Only the bands with 10 m resolution (blue, green, red, near-infrared) and 20 m resolution (four bands in the red-edge and two in the short-wave infrared) are be used. The 20 m resolution bands are resampled to 10 m using a bicubic interpolation with the appropriate spatial radius to avoid over-smoothing or aliasing.
Reference Data Sources and Nomenclature The reference dataset is composed of the same existing data sources and produced with the same procedure as described in  [8] , that is, a fusion of Corine Land Cover (CLC) 2012  [28] , the French National Geographic Institute "BD Topo"  [29] , the agricultural Land Parcel Information System "Registre Parcellaire Graphique" (RPG)  [30]  and the Randolph Glacier Inventory  [31]. Therefore, the nomenclature is composed of the following classes: CUF Continuous urban fabric; DUF Discontinuous urban fabric; ICU Industrial or commercial units; RSF Road surfaces; ASC Annual summer crops; AWC Annual winter crops; IGL Intensive grassland; ORC Orchards; VIN Vineyards; BLF Broad-leaved forest; COF Coniferous forest; NGL Natural grasslands; WOM Woody moorlands; BDS Beaches, dunes and sand plains; BRO Bare rock; GPS Glaciers and perpetual snow; WAT Water bodies.
Training and Test Data As usual in supervised classification, we randomly split the available reference data in 2 disjoint sets, one used for classifier training and the other one used to measure the quality of the classification using standard metrics. The performance evaluation is made over pixels. Since the reference dataset is composed of polygons where the pixels can be highly correlated (agricultural plots, a forest stand, etc. ), we ensure that training and test polygons are different, which avoids that two pixels from the same polygon, and therefore, likely very similar, are used one for training and the other for validation. This avoids optimistic accuracy estimations. For this work, 67% of the data were used for training and 33% for validation. The split was done so that these ratios are enforced for each image tile and class.
Due to the high level of sparsity of the labeled data, this classical network suffers from some limitations in our specific application. In our datasets, annotated polygons are sparse and do not necessarily include the boundaries between classes (e. g. , the edge between a cultivated field and a forest). As confirmed by our initial evaluation of the original U-Net architecture in Section 5, this leads to blurry, soft and erroneous class boundaries when applying the network in a dense prediction setting. In this section we introduce and discuss FG-UNET our modified (FG for Fine Grained) U-Net architecture that is adapted to our specific problem of land cover classification.
Time Series Handling The data used for this study are 33 Sentinel-2 images with 10 spectral channels for the entire year 2016. Two approaches of network design to handle this kind of data are possible: early fusion and late fusion. Early fusion supposes that the first layer takes as input N × N patches with C × T channels, where C = 10 spectral channels per patch and T = 33 temporal samples. Thus, a fusion of all these acquisitions at the beginning of the network would drastically increase the number of parameters in the first layers, where back-propagated gradients have low values and high levels of noise. The method chosen is a late fusion. The input is broken up into G temporal groups containing T/G temporal samples of C channels. The network takes as input one temporal group and outputs an intermediate representation.
Training Method To efficiently learn the proposed FG-UNET model, a training method was set up in order to take into account the specificities of satellite images, of the reference ground-truth data and of our HPC infrastructure. Indeed each Sentinel-2 tile used in this study has a size around 80 GB (16 bit encoding, 33 dates, 10 spectral bands, at 10 m resolution over 110 km × 110 km) which is much bigger than current images used in deep learning applications, labels used are very sparse and class labels are strongly imbalanced. Our HPC cluster provides 252 compute nodes with 24 CPU cores and 120 GB of memory per node, that is, a total of 6048 cores and 30TB of memory.
Preprocessing For faster convergence, CNN models need input data that has normalized values, stored as floating point values. Some works normalize by subtracting the mean and dividing by the standard deviation of each channel. Another approach, used here in order to normalize patches used during learning phase, is to compute the extreme values for each band. We use 3rd and 97th percentiles computed across the whole dataset instead of minimum and maximum values, due to outliers or missing pixels (no data due to orthorectification) potentially present in images. In this way, for each pixel (i, j) ∈ N × N of each channel c ∈ [0. 329] , each channel c of input patches p is normalized in the interval [0,1]: p c (i, j) = clip p c (i, j) -3 rd pct(c) 97 th pct(c) -3 rd pct(c) , 0, 1 (2) Reference data are available as polygons.
Class Imbalance Handling The frequency of the different classes in the Sentinel-2 dataset is heavily imbalanced, showing a ratio of 1-100 between the least and most frequent classes, as illustrated in Table  1. We compute class weights, used in Equation (2) as follows: weight k = n tot n k (4) Furthermore, during training, instead of choosing a random patch, we first randomly choose a class before choosing a random patch for this class. This increases the number of patches containing minority classes that are used for training.
Experimental Configuration To test our method, the hardware environment is an HPC infrastructure: a cluster on a low latency network (Infiniband) with shared GPFS storage. Each node has 24 CPU cores and 120 GB of memory. In total, there are 252 compute nodes available for a theoretical maximum of more than 6000 CPU cores. Our software environment is composed of CentOS 7 operating system, Python 3. 5 as programming language, the high-level neural networks API Keras with TensorFlow back end, and Horovod for a distributed training. Our code is publicly available online (DOI 10. 5281/zenodo. 1474224; https://github. com/vpoughon/RT_DL_OSO_public).
Baselines As mentioned previously, an operational processing chain based on the approach described in  [8]  is currently used to produce land cover maps. It uses Random Forests (RF), a state-of-the-art remote sensing image classifier. We use the same RF implementation to assess convolutional neural networks (CNN) method. It is provided by OpenCV through the supervised classification framework of Orfeo Toolbox 6. 2. Parameters used are 100 trees in the forest, a maximum depth of the tree of 20 and at least 10 samples in each node. For noise reduction, a regularization of generated map is performed using a structuring element with a radius of one pixel. This RF method is applied to the same dataset as the CNN approach. We introduce several other baselines: 1. MLP 3 layers: the 1 × 1 3 layer convolutional path of the FG-UNET model by itself.
Metrics and Their Limitations The statistical comparison presented here uses standard metrics: Cohen's Kappa coefficient as an overall measure and the F-Score coefficient (the harmonic mean between precision and recall). Although the use of Cohen's Kappa can be argued when comparing different datasets  [36]  we use it here to compare different methods on the same datasets. However, in the operational context, these metrics have some limitations. In the validation dataset used in these experiments (similar to  [8] ), pixels located in the center of spatially coherent areas such as agricultural plots or city blocks are far more common than pixels on the edges or in the corners. This implies that global scores like Overall Accuracy, Kappa, F-scores do not fully reflect the quality of the classification.
Indeed, the degradation of minority elements like edges and corners can be overshadowed by an improvement in the central areas. However, these negative effects are very visible in the output maps. For example, in Figure  8a , the fine details of the harbour area are smoothed out. The aim of this section is to show how this degradation can be quantified through a robust and fast measure that evaluates whether or not a contextual classification method respects the high spatial frequency areas in the image. If a dense testing dataset is available, validation samples can be split into separate categories that focus on the corners, edges, or fine elements, as is done in  [37]. However, in this study, the validation dataset is sparse, meaning that such an approach can not be used. Furthermore, the reference polygons are eroded to avoid errors due to misregistrations between satellite images and annotations.
• Error bars show that the variability of results is higher for FG-UNET than for RF. FG-UNET and RF learning and classification were performed five times, in order to analyze variability of models. The Kappa coefficient remains the same, but for some classes FG-UNET results may vary. This variation is mainly found for minority classes, for which the data augmentation used for training is not sufficient to capture the intra-class variability and therefore, the sensitivity to initialization is important. It is interesting to notice in Figure  6  that the more a tile contains artificial areas (see Table  1 ), the better are FG-UNET results. Indeed, tiles T30TYN and T31TGK are located on mountain areas, and T31UDQ (bigger gap between FG-UNET and RF) is located over Paris. This is due to the fact that contextual information (texture, mix of materials) is more important for urban classes than for vegetation, for which the temporal and spectral signatures convey more information.
Cross Domain Transferability It is interesting to compare the ability of RF and FG-UNET to classify a tile whose training dataset is not used for learning. To do so, for both RF and FG-UNET methods, learning is performed on training datasets of 10 tiles, and the classification is evaluated on the testing dataset of the 11th tile. Results are presented on Figure  19  and compared to the classical results (model learned on training dataset of 11 tiles). We observe on Figure  19:  • a drop in performance when a tile is not part of learning. Yet, it highly depends on the tile content. This drop is particularly high for mountain tiles (T30TYN and T31TGK) whose content is different from other tiles, and for T30TXQ which also has very few AWC labels. On the other hand, their is almost no drop for tile T30UXV.
The computation times shown in Table  2  are obtained using 11 tiles. If we wanted to map the European Union, we would need 450 tiles, and therefore, the training time would increase by a factor of more than 40. In this context, the ability to fine-tune a pre-existing model is very useful. Indeed, climate differences between years don't allow to use a previoulsy trained model on newly acquired data  [39]. While training a CNN model as FG-UNET for every map to be produced is too costly, fine-tuning the model has a cost comparable to the one of training a simpler classifier as RF. In the same way, using the already trained model as a (contextual) feature extractor and replacing the final fully connected layers by a more robust classifier as RF can also allow to combine the advantages of both approaches.
Future work following this study could focus, firstly, on reducing over-fitting and increasing generalization across different eco-climatic regions. One approach could be to optimise the architecture of the network through automatic methods such as Differentiable Architecture Search  [41]. Secondly, to alleviate the problems of sparse annotations, semi-supervised learning methods such as  [42]  could be employed. Furthermore, a more fair comparison with Random Forest should be performed. CNNs naturally capture a pixel spatial context due to their wide receptive field. However, random forest is only ever given a single pixel time-series as input. This makes the theoretical comparison of their classification performance slightly biased, because the two classifiers are not given the same information per data point.
Nonetheless, the comparison presented in this work remains valid as an empirical investigation of the capacities of the two systems in an operational context. Finally, our code should be benchmarked on a GPU cluster, which would improve training times, although at a high acquisition cost. Our code is available under an open-source license online (DOI 10. 5281/zenodo. 1474224; https://github. com/vpoughon/RT_DL_OSO_public).
Overall, our results for the parameters of GW170817 are consistent with, but more precise than, those in the initial analysis  [3]. The main improvements are (i) improved 90% sky localization from 28 deg 2 to 16 deg 2 without use of EM observations, (ii) improved constraint on inclination angle enabled by independent measurements of the luminosity distance to NGC 4993, (iii) limits on precession from a new waveform model that includes both precession and tidal effects, and (iv) evidence for a nonzero tidal deformability parameter that is seen in all waveform models. Finally, we analyze the potential postmerger signal with an unmodeled Bayesian inference method  [39]  using data from the Advanced LIGO detectors and the GEO600 detector  [40]. This method allows us to place improved upper bounds on the amount of postmerger GW emission from GW170817  [41].
A. Bayesian method All available information about the source parameters ⃗ ϑ of GW170817 can be expressed as a posterior probability density function (PDF) p( ⃗ ϑjdðtÞ) given the data dðtÞ from the GW detectors. Through application of Bayes' theorem, this posterior PDF is proportional to the product of the likelihood L(dðtÞj ⃗ ϑ) of observing data dðtÞ given the waveform model described by ⃗ ϑ and the prior PDF pð ⃗ ϑÞ of observing those parameters. Marginalized posteriors are computed through stochastic sampling using an implementation of Markov-chain Monte Carlo algorithm  [43, 44]  available in the LALINFERENCE package  [45]  as part of the LSC Algorithm Library (LAL)  [46]. By marginalizing over all but one or two parameters, it is then possible to generate credible intervals or credible regions for those parameters.
B. Data For each detector, we assume that the noise is additive, i. e. , a data stream dðtÞ ¼ h M ðt; ⃗ ϑÞ þ nðtÞ where h M ðt; ⃗ ϑÞ is the measured gravitational-wave strain and nðtÞ is Gaussian and stationary noise characterized by the one-sided power spectral densities (PSDs) shown in Fig. 1. The PSD is defined as S n ≡ ð2=TÞhj ñðfÞj 2 i, where ñðfÞ is the Fourier transform of nðtÞ and the angle brackets indicate a time average over the duration of the analysis T, in this case the 128 s containing the dðtÞ used for all results presented in Sec. III. This PSD is modeled as a cubic spline for the broadband structure and a sum of Lorentzians for the line features, using the BAYESWAVE package  [39, 47] , which produces a posterior PDF of PSDs. Here, we approximate the full structure and variation of these posteriors as a point estimate by using a median PSD, defined separately at each frequency.
The analyses presented here use the same data and calibration model for the LIGO detectors as Ref. [3] , including subtraction of the instrumental glitch present in LIGO-Livingston (cf. Fig. 2  of Ref. [3] ) and of other independently measured noise sources as described in Refs. [48] [49] [50] [51]. The method used for subtracting the instrumental glitch leads to unbiased parameter recovery when applied to simulated signals injected on top of similar glitches in detector data  [52]. The data from Virgo has been recalibrated since the publication of Ref. [3] , including the subtraction of known noise sources during postprocessing of the data, following the procedure of Ref. [38]  (the same as described in Sec. II of Ref. [53] ). While the assumption of stationary, Gaussian noise in the detectors is not expected to hold over long timescales, our subtraction of the glitch, known noise sources, and recalibration of the Virgo data helps bring the data closer to this assumption.
Applying the Anderson-Darling test to the data whitened by the on-source PSDs generated by BAYESWAVE, we do not reject the null hypothesis that the whitened data are consistent with zero-mean, unit-variance, Gaussian noise N ð0; 1Þ. The test returns p-values > 0. 1 for the LIGO detectors' data. Meanwhile, the test is marginal when applied to the Virgo data, with p ∼ 0. 01. However, the information content of the data is dominated by the LIGO detectors, as they contained the large majority of the recovered signal power. The results of the Anderson-Darling tests support the use of the likelihood function as described in Ref. [45]  for the signal characterization analyses reported in this paper. The measured strain h M ðt; ⃗ ϑÞ may differ from the true GW strain hðt; ⃗ ϑÞ due to measured uncertainties in the detector calibration  [54, 55].
A. Localization For most of the analyses in this work, we assume a priori that the source of GW170817 is in NGC 4993. However, the improved calibration of Virgo data enables better localization of the source of GW170817 from GW data alone. To demonstrate the improved localization, we use results from the updated TaylorF2 analysis (the choice of model does not meaningfully affect localization  [105] ), shown in Fig. 3. We find a reduction in the 90% localization region from 28 deg 2  [3]  to 16 deg 2. This improved localization is still consistent with the associated counterpart SSS17a/AT 2017gfo (see Fig. 3 ). For the remainder of this work, we incorporate our knowledge of the location of the event. While fixing the position of the event to the known location within NGC 4993, we infer the luminosity distance from the GW data alone.
For the two EOB waveforms, the 90% highest posterior density credible interval for Λ is (0,560) for SEOBNRv4T and (10,690) for TEOBResumS. For SEOBNRv4T, the posterior for Λ has a peak away from Λ ¼ 0, and the lower bound of Λ ¼ 0 is not simply due to the prior bound. In fact, the value of the posterior distribution is the same at both the upper and lower limits of the 90% credible interval, indicating that the peak is resolved. Recently, De et al. performed an independent analysis of the GW data to measure the tidal parameters  [141]. Their results are broadly consistent with those presented here but are made under the assumption that the two merging NSs have the same EOS. They assume that the two NSs have identical radii and that the tidal deformabilities of the individual stars are related by the approximate relation Λ 1 ¼ q 6 Λ 2 , whereas we allow the tidal parameters to vary independently.
The priors on the wavelet quality factor and phase are flat in (0,200) and ð0; 2πÞ, respectively. The priors on the central frequency and time are determined by the analysis duration and bandwidth described below, while the amplitude prior is determined through the SNR of each wavelet and discussed in more detail in Ref. [39]. We use the analysis described in Ref. [150]  to estimate an upper bound on the amplitude of a putative GW signal assumed to be present but at insufficiently high SNR to generate a statistically significant detection candidate. We use coincident data from the two LIGO detectors and from GEO600  [40] , which has comparable sensitivity to Virgo at high frequency. Indeed, the sky location of GW170817 is particularly favorable for the GEO600 antenna response so that any high-frequency signal component observed by GEO600 will have a SNR greater than or equal to that expected in Virgo.
During this period, the Virgo data above 2 kHz suffer from an abundance of spectral lines and transient noise and, therefore, are not included in this analysis. It should also be noted that GEO600 was not in science mode due to investigations into a degraded squeezer phase error point signal leading to a reduced level of squeezing. At the time of the event, the investigations were passive observations. Otherwise, GEO600 was in nominal running condition. The calibration of the LIGO detectors is more uncertain above 2 kHz than at lower frequencies, but it is still within 8% in amplitude and 4 deg in phase  [54]. The GEO600 calibration uncertainty is estimated to be within 15% in amplitude and 15 deg in phase in the 1-4 kHz band. GEO600 was not used in a previous search for high-frequency GW emission due to an insufficient characterization of data quality and analysis tuning, which would have been required for accurate background estimation  [41].
The analysis is performed over the 1024-4096-Hz band, which is sufficient to contain the full postmerger spectrum. We determine the relative evidence for two models: that the on-source data are described by Gaussian noise only, or by Gaussian noise plus a GW signal as described in Refs. [151, 152]. We find that the Gaussian noise model is strongly preferred, with a Bayes factor (evidence ratio) of 256. 79 over the signal model. This result is consistent with both prompt collapse to a BH and with a postmerger signal that is too weak to be measurable with our current sensitivity. We further characterize the absence of a detectable signal by forming 90% credible upper limits on three measures of signal strength: (i) the network SNR, evaluated over 1-4 kHz, (ii) the strain amplitude spectral density (ASD), and (iii) the spectral energy density (SED).
We compute the 90% credible upper limit on the network SNR directly using the reconstructed waveform posterior. We exclude signal power in our analysis band with ρ net > 6. 7 at the 90% level. The top panel of Fig. 13  reports the upper limits and expectations for the strain ASD induced in the LIGO-Hanford instrument. These limits are formed directly from the posterior probability distribution for the reconstructed waveform in the 1 s of data around the merger time measured from the premerger observations using a coherent analysis of data from all three detectors. The noise ASDs for each instrument are shown for comparison. As one would expect in the absence of a signal, the upper limits on strain amplitude approximately follow the shape of the noise spectrum. We also overlay a small set of spectra obtained from simulations of BNS mergers using different EOSs with extrinsic parameters (i.