dataesr/scientific-paragraphs-categorization huggingface dataset:
Finally, given the high performance of the DECam and HSC pipelines, the corresponding images are perfect candidates for our non-contaminated datasets. These two pipelines not only efficiently detect but also interpolate problematic pixels (in particular saturated pixels, hot and bad pixels, cosmic ray hits). Such interpolations being a feature of several modern pipelines (e. g. , various NOAO pipelines, but also the LSST pipeline), we choose to treat these pixels as regular pixels so that the networks are able to work with images originating from such pipelines. Patches of size 400 × 400 pixels are randomly extracted from the cleaned images. 75% of them are used to generate training data and the remaining 25% for test data. M. Paillassa, E. Bertin, H. Bouy: MaxiMask and MaxiTrack  The final non-contaminated dataset includes 50,000 individual images, ensuring that we have a sufficiently diverse and large amount of training data for our experiment.
A non representative training set can severely impact the performance of a CNN and result in significant biases in the classification task. To prevent this, we measure a number of basic properties describing prototypical aspects of ground-based astronomical images to verify that their distributions in the uncontaminated dataset are wide enough and reasonably well sampled. The measured properties include, for example, the average full-width at half-maximum (FWHM) of point-sources is estimated in each image using PSFEx  (Bertin 2013). This allows us to ensure that the training set covers a broad range of ambient (seeing) conditions and point spread functions (PSFs) sampling. Also, the source density (number of sources in the image divided by the physical size of the image) is measured to make sure that our training set encompasses a broad range of source crowding, from sparse cosmological fields to dense, low-galactic latitude stellar fields.
Generating training samples Both types of contaminants -global and local contaminantsmust be handled separately: they require different neural network architectures, and different training data sets as well. Figure  7  gives a synthetic view of the sample production pipeline and the various data sources. The breakdown per imaging instrument of the COSMIC DANCe dataset is listed Table  3. The following subsections treat about some special features of the sample generation. 2 Including astrophysical sources in the "background" class can seem somewhat counter-intuitive in a purely astronomical context, but for consistency we choose to follow the computer vision terminology and meaning.
Contaminant classes are so statistically insignificant (down to one part in 10 6 with real data, typically) that the classifier may be tricked into assigning all pixels to the background class. To prevent this, we start by applying a basic weighting scheme to each pixel according to its class representation in the training set, that is each pixel p of batch image b belonging to classes in C p,b is weighted by w p,b defined as w p,b = ω c ∈C p,b w c , (14) with w c =        P(ω c |T ) i 1 P(ω i |T )        -1 , ( 15 ) where P(ω c |T ) is the fraction of pixels labeled with class ω c in the training dataset T. The P(ω c |T )'s do not sum to one as many pixels belong to several classes and are thus counted several times. We find that the weighting scheme brings slightly better results and less variability in the training if weights are computed at once from the class proportions of the whole set, instead of being recomputed for each image.
However with strongly imbalanced datasets, such as our pixel data, one must be very cautious with the T PR, FPR and ACC values for assessing the quality of the results. For example, if one assumes that there are 1,000 pixels of the contaminant class (P) and 159,000 pixels of the background class (N) in a 400×400 pixel sub-image, a T PR of 99% and a FPR of 1%, corresponding to an accuracy of 99%, would actually represent a poor performance, as it would imply 990 true positives, 10 false negatives, 157,410 true negatives, and 1590 false positives. In the end, there would be more false positives FP (pixels wrongly classified as contaminated) than true positives T P. For this reason the ROC curves in Fig. A  A. 1 are displayed with a logarithmic scale on the FPR axis. We require the FPR to be very low (e.
Robustness regarding the context The MaxiMask neural network is trained using mostly images that include all contaminant classes. Hence, we must check if the network performs equally well independently of the context, that is if it delivers equally good results for images containing, for example, a single class of contaminant. To this aim, for every contaminant class, we generate a dataset of 1,000 images affected only by this type of contaminant (except saturated and background pixels), and another dataset of 1,000 images containing only saturated and background pixels. We then compare the performance of MaxiMask for each class with the that obtain on the corresponding dataset. We find that performance (AUC) is similar or even slightly higher for the majority of the classes. This shows that the network is not conditioned to work only in the exact context of the training.
The results are presented in Table  7. As it can be seen, for all classes but fringes and nebulosity, performance improves when a single type of contaminant is present. The slight improvement may come from the fact that ambiguous cases (when pixels are affected by more than one To do so, we generate two datasets containing only the cosmic ray contaminant class (plus object and background). A well sampled set of images with FWHMs larger than 2. 5 pixels, and an undersampled image set with FWHMs smaller than 2. 5 pixels. We run MaxiMask and the Astro-SCRAPPY Python implementation LA Cosmic. To make a fair comparison, LA Cosmic masks are dilated in the same way as the ground truth cosmic ray masks of MaxiMask. However, while MaxiMask generates probability maps that can be thresholded at different levels, LA Cosmic only outputs a binary mask.
Using MaxiMask and MaxiTrack MaxiMask and MaxiTrack are available at https://www. github. com/mpaillassa/MaxiMask. MaxiMask is a Python module that infers probability maps from FITS images. It can process a whole mosaic, a specific FITS image extension, or all the FITS files from a directory or a file list. For every FITS file being processed a new FITS image is generated with the same HDU (Header Data Unit) structure as the input. Every input image HDU has a matching contaminant map HDU in output, with one image plane per requested contaminant. The header contains metadata related to the contaminant, including the prior and threshold used. An option can be set to generate a single image plane for all contaminants, using a binary code for each contaminant. Such composite contaminant maps can easily be used as flag maps, for example, in SExtractor.
Based on command line arguments and configuration parameters, one can select specific classes, apply updates to the priors and thresholds to the probability maps. The code relies on the TensorFlow library and can work on both CPUs or GPUs, although the CPU version is expected to be much slower: MaxiMask processes about 1. 2 megapixel per second with an NVidia Titan X GPU, and about 60 times less on a 2. 7GHz Intel i7 dual-core CPU. Yet, there is probabily room for improvement in processing efficiency for both the CPU and GPU versions. MaxiTrack is used the same way as MaxiMask, except that the output is a text file indicating the probability for the input image(s) to be affected by tracking errors (one probability per extension if the image contains several HDUs). It can also apply an update to the prior.
Both effects are a significant source of nuisance in wide field exposures, especially in crowded fields and deep images, where they generate false, transient sources, and can affect high precision astrometric and photometric measurements. Another category of common issues is defocused or excessively aberrated exposures, as well as trails caused by charge transfer inefficiency, all of which which could easily be implemented in MaxiTrack. Also, the training set used in the current version of Max-iMask and MaxiTrack does not include images from spaceborn telescopes nor, more generally, diffraction-limited imagers. Therefore, they are unlikely to perform optimally with such data, although limited testing indicates that they may remain usable for most features, an example of prediction on HST data is shown in Figs.
Acknowledgements. M. P. acknowledges financial support from the  Centre National d'Etudes Spatiales (CNES)  fellowship program. We are grateful to  Mike Read , of the  Royal Observatory, Edinburgh , for providing us with data from the UKIRT telescope, and to  Vincent Lepetit  for providing comments and suggestions that helped improve the paper. This research has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation programme  (grant agreement No  682903 ,  P. I. H. Bouy ), and from the  French State  in the framework of the  "Investments for the future" Program ,  IdEx Bordeaux , reference  ANR-10-IDEX-03-02. We gratefully acknowledge the support of  NVIDIA Corporation  with the donation of one of the Titan Xp GPUs used for this research.
Funding was contributed by the  FIRST program  from  Japanese Cabinet Office , the  Ministry of Education, Culture, Sports, Science and Technology (MEXT) , the  Japan Society for the Promotion of Science (JSPS) ,  Japan Science and Technology Agency (JST) , the  Toray Science Foundation ,  NAOJ ,  Kavli IPMU , KEK, ASIAA, and  Princeton University. This paper makes use of software developed for the  Large Synoptic Survey Telescope. We thank the  LSST  Project for making their code available as free software at http://dm. lsst. org. The Pan-STARRS1 Surveys (PS1) have been made possible through contributions of the  Institute for Astronomy , the  University of Hawaii , the Pan- STARRS  Project Office, the  Max-Planck Society  and its participating institutes, the  Max Planck Institute for Astronomy, Heidelberg  and the  Max Planck Institute for Extraterrestrial Physics, Garching , The  Johns Hopkins University, Durham University , the  University of Edinburgh, Queen's University Belfast , the  Harvard-Smithsonian Center for Astrophysics , the  Las Cumbres Observatory Global Telescope Network Incorporated , the  National Central University of Taiwan , the  Space Telescope Science Institute , the  National Aeronautics and Space Administration  under Grant No.
NNX08AR22G  issued through the  Planetary Science Division of the NASA Science Mission Directorate , the  National Science Foundation  under Grant No. AST-1238877 , the  University of Maryland , and  Eotvos Lorand University (ELTE)  and the  Los Alamos National Laboratory. Based on data collected at the  Subaru Telescope  and retrieved from the HSC data archive system, which is operated by  Subaru Telescope  and Astronomy Data Center at  National Astronomical Observatory of Japan. This paper includes data gathered with the Swope telescope located at  Las Campanas Observatory, Chile. Based on observations obtained with  MegaPrime/MegaCam , a joint project of CFHT and CEA/DAPNIA, at the  Canada-France-Hawaii Telescope (CFHT)  which is operated by the  National Research Council (NRC)  of  Canada , the  Institut National des Science de l'Univers of the Centre National de la Recherche Scientifique (CNRS) of France , and the  University of Hawaii.
Reproducible research is needed to ensure that scientific results in the literature are reliable, unbiased, and verifiable by others. The journal Image Processing On Line (IPOL) publishes reproducible articles since 2010. This means publishing an algorithm by a literary description, a pseudo-code, its source code, a series of test examples, an online facility allowing to test the code on this data and other data submitted by the user, and finally an experimental archive. In this work we discuss how to publish and review reproducible research in the specific discipline of remote sensing. We put a special emphasis on the construction and proper documentation of public datasets. We show case studies of remote sensing articles publicly available in IPOL, which demonstrate the feasibility of reproducible research in this area.
In IPOL this is considered as a whole and text + source code + data is published under the same DOI. Since the credibility crisis in research was acknowledged, several initiatives have flourished  [5] , as for example the OpenAIRE 2  program operated by CERN, and Zenodo  [6]  (a general-purpose and open-access repository under openAIRE), as well as infrastructure and services that can be used for reproducible research such as the Galaxy platform  [7]  in the field of genomic research, or IEEE's Code Ocean service  3  , among others. In  [8]  the authors propose a badging system to identify works that comply with minimum requirements of reproducibility and replicability in Remote Sensing research. The RSRR badge would be based on about 20 criteria concerning web page, code, and data. It ends up requiring actions from the authors that are similar to those requested in IPOL.
In this work, based on the experience of the last ten years developping the IPOL journal, we give recommendations to authors and editors willing to publish reproducible research in remote sensing, and show real case studies demonstrating that reproducible research in remote sensing is certainly feasible (and absolutely necessary). The plan of the article follows. Sec. II discusses how to write and review reproducible articles in remote sensing, and gives some details on how this is conducted in IPOL. Sec. III addresses the problem of public datasets, their availability, their quality, and how to re-use them to establish a reliable state of the art by proper evaluation and scoring of methods. Sec. IV shows five case studies of reproducible remote sensing published in IPOL, giving details on the application of the methods and on how the datasets were built.
II. HOW TO WRITE AND REVIEW A REPRODUCIBLE RESEARCH IN REMOTE SENSING Reviewing a reproducible research article in remote sensing requires additional steps with respect to a classic publication in order to ensure the reproducibility of the results, repeatability of the experiments, availability of the data and, in general, the overall quality of the published works. A fundamental step, often forgotten in non-reproducible publications, is to check that the source code and the algorithmic descriptions (pseudo-codes) in the article match accurately. Ideally, one should be able to re-implement the method exactly from the descriptions in the article. However, both the source code and the pseudo-codes in the article need to be provided, because the source code shows precisely which operations are performed, while the pseudo-codes are much shorter and are the code' specification, that also allows rewriting in another language.
They also help the reader focus on the significant parts of the code. The article itself puts everything in its scientific context. Establishing a true state of the art in remote sensing methods requires reliable datasets. By reliable we mean that the process that built them is detailed and communicated to the community (say, as a publication in a journal). The dataset should not be biased and might contain real data as well as synthetic. Examples of biased datasets would be cloudfree Sentinel-2 images or SMOS data without radio-frequency interference (RFI)  [9] ; indeed, about 60% of SMOS data is affected by RFIs and a similar proportion of Sentinel-2 images are cloudy  [10]. The availability of proper datasets allows for a fair comparison and ranking of algorithms. Thus, the editor and reviewers of a reproducible article in remote sensing should request the use of public and well-documented datasets.
Sec. III discusses in detail this topic. Finally, the editors should encourage that the authors to show not only positive results, but also cases of failure to better understand the limitations of the method. Here, IPOL's setup comes to great advantage, as it allows referees to test the paper's code on their own data. These are minimal recommendations for reviewing reproducible articles. In Sec. II-A we describe how the review process is conducted in the IPOL journal. A. The IPOL approach IPOL was born as a fully reproducible research journal  [11] ,  [12]. After more than ten years of experience, we have implemented standard procedures to ensure reproducible publications beyond the minimal requirements mentioned before  [13]. Each IPOL article needs to follow the principles of reproducibility (the possibility that other researchers can obtain the same results given that source code and data is public and comes along a detailed scientific article) and repeatability (the fact that the experiments can be re-done as many times as needed, with publicly available evaluation datasets).
Indeed, all source code in IPOL is open-source, data is publicly available to the scientific community (with details on how datasets were built), and the text (the article) is under public documentation license. Repeatability is ensured by providing public demonstrations that allow users to freely test the algorithms with the proposed datasets or their own data. A public archive of experiments is also available. The access is open-source and there is no author or user fee. The IPOL demo editors take care of building the demos of the authors, who are only requested to give their source code, the compilation instructions, examples of data they would like their demo to offer, and a description of the demo. IPOL has style guidelines for authors about the writing of the article 4  as well as for the software  5.
At first the software guidelines were a list of highly-technical mandatory requirements, but soon it became clear that only experts in software engineering were able to follow them; we realized that they were preventing authors to submit their work. Thus, we adopted the practice of establishing a tight contact between editors and authors to reach the required source code quality. This approach has proven successful and indicated that enforcing the adoption of high-quality coding standards is not automatic and is in fact an educational process. A major problem when reviewing research in computational sciences is the requirement of finding reviewers and editors who are both competent in analyzing source code and are experts in the corresponding scientific field. The lack of these multi-disciplinary profiles makes hard the in-depth review of reproducible articles.
In IPOL this problem has been mitigated by choosing at least two reviewers, one of them an expert in software engineering and the other an expert in the scientific field. Note that we require that both reviewers (as well as the editor) are skilled in both. The most important task of the code reviewer is certainly to check that the source code matches accurately the algorithmic descriptions given in the article. With this review system, the typical review time from submission to the edition of an online preprint is two weeks. The time from preprint to publication has high variance, as many authors often work on improvements. It is on average four months. Once the article, code, and data have been published, all of them become available as a whole under the same DOI. If any bugs are detected after publication  6  , the authors are requested to submit a revised version of their code, which will be checked by the IPOL's Editorial Board (in case of a bug correction, there is no need to start another peer-review evaluation), and added to the history which is available in the article's page  7.
Note that even if corrections are performed after publication, the original sources are always kept unaltered and under the same DOI. Here we refer to bugs or technical changes, but keeping the same algorithm. In case the method changes, also would the pseudo-code descriptions and the implementation, and thus it would require indeed a new submission given that it would correspond to a different algorithm. Hence, if the improvements are significant and the changes major, authors are encouraged to submit them as a different publication. Each IPOL article comes along with an online demo allowing to quickly test the algorithms on sample data or user provided data. Also, each demo keeps an archive of experiments performed with uploaded data. This has proven very useful to have a large corpus of use cases and to uncover the interests of the community.
Indeed, demos with large archives mean a large interest (some articles have archives of up to 59000 distinct experiments). IPOL encourages authors to restrict as much as possible the code dependencies on external libraries. On the one hand, this ensures that the main code is self-contained and that its functionality is not black-boxed inside external components. On the other hand, it helps long-term durability of the code. There is no guarantee that a published code can be compiled in the long term, given that it depends on external libraries that can change the signature of the functions, as well as the compilation rules. In the case of Python code, the IPOL demo system automatically takes care of creating virtual environments  [14]  and it is mandatory that authors declare in a requirements.
COMPARISON Establishing the true state of the art in remote sensing research requires a fair comparison of algorithms. This implies reliable datasets for evaluation purposes, and raises the question of what reliable datasets are, and how to build them. Depending on the application, the dataset could include ground-truth data, that is, a reference to compare with. For example, let us consider a de-noising algorithm for satellite images. If the noise model is known (e. g. , Poisson noise with known parameter), one could evaluate the de-noising algorithms by adding synthetic noise to a set of noise-free groundtruth images used as reference. How to build these noise-free images? They could be obtained by averaging several snapshots of the same location and conditions (that is, including same incidence angle, season, time, atmospheric conditions, calibration of the sensor, no changes on the ground).
It is important that the datasets are publicly available and that the details on how they were built are published. This helps in assessing any possible limitation in the data or in the groundtruth annotation. In the case of deep-learning applications, the size of the dataset is also a requirement. We can cite two actual examples of publication of datasets in IPOL, such as high precision 3D data sets  [16] , and human physiological data  [17]. In the case of IPOL, we have standardized the concept of templates in the demo system. They are in fact datasets of objects than can be used by different demos. So far there are 23 different datasets that can be added to any IPOL demo, including denoising (noise-free images), microtextures, standardTestImages (standard test images used in the image processing literature), JPEGquality (images under different JPEG quality factors, for compression analysis), forgery (image forgery detection), deblurring, statokinesigrams (physiological data), kodakImageSuite (standard dataset), CloudDetection_Sentinel2_L1C_RGB and CloudDetection_TimeSeries_Sentinel2_L2A (Sentinel-2 datasets for cloud detection).