dataesr/scientific-paragraphs-categorization huggingface dataset:
Conclusions In this paper, we refer to the porting of Rulex, a machine learning software that natively runs on Windows 64 Bits, on the Raspberry Pi, for Edge Computing applications. We also reported the results obtained in different application domains, namely with five unrelated datasets, which we used to test the performance of our implementation in real IoT environments. The main result obtained is that Rulex now operates in a Client/Server setup with the interface being operated on a PC as a Windows 32 Bits application, with the machine learning Algorithms being run on the Raspberry Pi ARM 32-Bit microcontroller. As regards the performance obtainable through this implementation, first we considered a dataset related to the classification of pedestrians and vehicles using a high-frequency radar: An ML workflow was implemented on our platform and good results were achieved.
However, as such dataset is relatively small, we considered four additional datasets from diverse application fields. Three pre-processed datasets having a larger number of samples and with low and high dimensionality and varying skewness were adopted. The pre-processed datasets include a Human activity Recognition dataset using Smartphones, a vehicle activity detection dataset, and an EEG Classification dataset related to mental state. Furthermore, a novel pre-processing Algorithm was developed and implemented in Python, that converts images into Time-series to pre-process a gender detection dataset that contains inconsistent facial images in terms of the background and dimensions of the image itself and the age, ethnicity, and pose of the subject. Also, the accuracy achieved for gender classification is considerably competitive with the literature, where the competition is dominated by mug-shot and forward-facing images.
Data Availability Statement: The Human Activity Detection Dataset using Smartphones can be found in the Kaggle online repository through the following address: https://www. kaggle. com/ uciml/human-activity-recognition-with-smartphones. The Mental Sate Detection Dataset can be found in the Kaggle online repository through the following address: https://www. kaggle. com/ birdy654/eeg-brainwave-dataset-mental-state. The Vehicle Activity Recognition Dataset can be found in the Kaggle online repository through the following address: https://www. kaggle. com/ smartilizer/commercial-vehicles-sensor-data-set. The image-based Gender Classification Dataset can be found in the Kaggle online repository through the following address: https://kaggle. com/ ashwingupta3012/male-and-female-faces-dataset.
Convolutional Neural Networks (CNN) have also been recently used for bias corrections to improve a 7 days air quality forecast issued from a chemistry transport model  [20]. Other recent works showed the possible use of NN to act as drop-in chemical solvers with orders-of-magnitude performance gain  [21, 22] , though error propagation over long time periods needs to be addressed. Following these previous works, this study proposes a novel approach to design a NN-based emulator of high-resolution air pollution simulations which are highly time-consuming. In  [23] , they showed that a very simple regression with an adequate selection of input variables may relevantly emulate a CTM and mimic the behavior with identical performances. Though, this preliminary study was made on daily datasets over a large period.
Here, we introduce a new CTM-NN strategy and base the training on a shorter period with hourly data. Compared with previous works, the novelty of our approach lies in the input data selection and the designed NN architectures. The use of an hourly dataset is also new compared to the previous study allowing a shorter period for the training phase and get a better representation of the evolution of dispersion conditions. We report a quantitative evaluation on a representative test case, which demonstrates the relevance of the proposed CTM-NN approach. At this stage an evaluation on PM 10 , PM 2. 5 and NO 2 , ozone is not addressed and it will be in a follow-up study by introducing more physics in the NN approach. We further discuss possible applications and improvement for operational uses.
The WRF simulation (WRF 3. 7. 1 version) used for the meteorology is nudged with NCEP (National Centers for Environmental Prediction) final analysis from GFS (Global Forecast System) meteorological fields at 1 o × 1 o. The 6-hourly time resolution version is used at the coarsest model initial and domain boundaries (ds083. 2 dataset  [30] ). For this study, the set-up and domains are rigorously the same depicted in a previous work aiming at simulating the air quality at fine resolution over the French Alps  [31]  providing CTM outputs for the meteorology and concentrations fields. Two different domains are defined as: (i) EUR01 covering a large part of Western Europe and (ii) over ALP0033 for the whole French Alps with detailed characteristics are detailed in table  1. The Alps domain (APL0033) encompasses an area from the Lyon municipality on the West part to the Leman Lake on the north to the Piemonte region in Italy on the East Part with a resolution of about 3 km (figure  2 ).
Grenoble and the Arve valley from Geneva to Chamonix are known to be air pollution hot spots in France due to their location in deep valleys with frequent stagnant cold meteorological conditions in wintertime. The HR simulation is performed over the ALP0033 and the LR simulation over the EUR01 domain and interpolated over the ALP0033 so that the ALP0033 mesh will be the working grid for the neural networks. The simulation was performed from 2013-11-15 01:00 UTC to the 2021-12-21 00:00 UTC with a spin-up period from the begin of November to ensure a good initialization. Therefore 864 hours are available to train (432 hours) and evaluate (432 hours) the various neural network strategies. The available observation dataset for the model evaluation is reported in  [31]  including only rural and urban background stations (figure  2 ).
This framework can be extended to spatio-temporal CNN even though we believe that for this super-resolution task, a spatial formulation of the problem is satisfactory enough. This is still an interesting idea though if the same work has to be achieved on data with high missing data rates, such as remote sensing and/or in situ dataset  [32]. Here, a simple CNN architecture is first considered in which the inputs are the same covariates used in the MLP, but stacked here in addition of the coarse resolution C as supplementary channels. The CNN architecture comprises a first hidden layer with 128 Conv2D 3 × 3 filters + ReLU activation (Rectified Linear Unit) with a batch normalization, followed by a second hidden layer with 64 5 × 5 filters + ReLU activation. The final layer maps the outputs of the second hidden layer to the required HR resolution by a single linear Conv2D 3 × 3 filter, see figure  5.
Regarding the training strategy, we use an Adam optimizer with a batch size of 4 through 100 epochs on a Microsoft Azure Virtual Machine (VM) powered by NVIDIA Tesla K80 with a GPU memory of 12GiB. The training time of a single MLP is only of 15 seconds, but without any parallelization, the whole training on the 7038 grid cells of the ALP0033 domain is about 26 hours. The same number of epochs is used for MLP, CNN and RCAN, though the RMSE loss function stabilizes between 50 and 100 cycles depending of the architecture. No major overfitting has been observed for this number of epochs though this is the case if increasing the number of cycles up to 200. The CNN-based architectures provide faster training procedures with only 50 minutes for the basic CNN and around 6 hours for RCAN. In the end, if the training time can significantly differ according to the NN architecture, their application on new datasets only takes a few seconds.
Results In this section, we report the results obtained for NO 2 and PM 2. 5 with the three NN-based super resolution strategies. We use the first half of the dataset for training: 432 hours ranging from 2013-11-15 01:00 UTC to 2013-12-03 00:00 UTC and the other half for validation, ending on 2013-12-21 00:00 UTC. Complementary results are provided in appendix A (figure A1, A2, A3 and A4) for PM 10 and the evaluation on an hourly basis in tables B1, B2 and B3 of appendix B. Figure  C5  of appendix C provides an evaluation on Δc for each NN. The definition of evaluation metrics is provided in appendix D. Typical patterns of high concentrations are observed along the road traffic network and urbanized areas with the original high resolution (HR). PM 2. 5 concentrations maps are smoother since PM emissions are more spread over rural areas particularly due to wood burning and long range transport of such species.
It is clear that the impact of the emission covariates is here largely overestimated in this situation with low PM 2. 5 pollution levels along the roads. This issue directly relates to the learning issue of optimizing the RMSE loss function in average over this training period. It might be more efficient to consider other loss functions and training datasets to address the issue of learning how the HR behaves in specific conditions for operational applications. As in  [23]  it is interesting to compare the HR, LR and the three NN simulation outputs with real observation data. For the observations available in the ALP0033 domain along the validation period, we provide in figures 10 and 11 additional Taylor diagrams for the comparison of HR, LR, and the three SRNN to daily-averaged rural and urban observations.
Discussion The best NN architecture RCAN is able to reproduce the behavior of the raw HR simulation with satisfactory performances. For some pollutants the NN model is even able to provide better results probably by smoothing some aberrant values calculated by the CTM during very stable situations leading to unrealistic concentration peaks. With minor improvements on ozone chemistry and the use of observational data to constrain the system similarly to MOS (Model Output Statistic) techniques or using a CNN approach for bias corrections  [20] , our approach can be quickly deployed for air quality forecasting. Once the training is performed the forecasting chain could deliver a forecast in a few second instead of hours. Then, it should be further investigated if a generic NN-based model is sufficient or if it has to be adapted to specific meteorological conditions.
Conclusion In this study, we have developed and evaluated the relevance of neural-network super-resolution approaches to downscale coarse chemistry transport modeling simulations focusing on three criteria air pollutants: NO 2 , PM 2. 5 and PM 10. These learning-based techniques take advantage of (i) fast coarse simulation outputs from the CHIMERE CTM embedding complex mathematical representations of physics and chemistry and particularly the long-range transport of pollutants, and (ii) more local features which are retrieved by neural network approaches. The reported quantitative and qualitative evaluation against both the high resolution reference simulation and real observation datasets support the relevance of the neural-network-based downscaling for the operational monitoring and forecasting of air quality.
) and 6(b) respectively show the NO 2 and PM 2. 5 mean concentrations over the validation period. CHIMERE high-resolution (top left panel) is the target that the super resolution NN operator aims at reconstructing, starting from the coarse resolution (top right panel) and additional covariates as input datasets. On the bottom panel, the maps obtained with the three NN-based super resolution architectures are displayed: from left to right MLP, CNN and RCAN. They all exhibit a significant improvement as compared to the coarse simulation. The main patterns are well reproduced by the three NN.
Acknowledgments This research was funded by the  DGA (French Directorate General of Armaments ; grant no. 2018 60 0074 ) in the framework of the  NETDESA  project. It was also supported by  ANR Projects OceaniX. It benefited from  HPC  and  GPU resources from Azure (Microsoft EU Ocean awards ) and from  GENCI-IDRIS  (Grant  2020-101030 ).
Description of ACT interface The routine production of ACT relies on two main steps. First the 12 training scenarios corresponding to the current air quality forecast are simulated on a high-performance computer with the CHIMERE model with a setup inspired from the French Air Quality Platform Prev'Air  (Rouïl et al. , 2009)  and the European CAMS regional production  (Marécal et al. , 2015). Then the surrogate model is automatically calibrated and exported to a web interface using the shiny package of the R language. An annotated screenshot of the ACT web interface is given in Fig. S9  of the Supplement. First, the pollutant of interest is selected in the list of compounds for which the surrogate has been validated: PM 10 , PM 2. 5 , NO 2 , O 3 max and O 3 avg , all of which are daily mean values, except for O 3 max which is the daily maximum level.
Code availability. The script used for the operational daily training of the ACT surrogate is available at https://github. com/ acolette/ACT_v1. 0 (last access: 15 February 2022) (https:// doi. org/10. 5281/zenodo. 5973299, Colette, 2022). The underlying CHIMERE chemistry-transport model is available at https://www. lmd. polytechnique. fr/chimere/chimere. php (Mailler et al. , 2017). Data availability. The modelling results used in the present study are archived by the authors and can be obtained from the corresponding author upon request. Supplement. The supplement related to this article is available online at: https://doi. org/10. 5194/gmd-15-1441-2022-supplement.
False positives can be introduced for biological reasons  [11, 12]  and through peak callers (FDR of 1-5% or more  [13] ). Besires errors, anomalous peaks can be caused by other biological and technical specificities (e. g. , different protein fixation kinetics), systematic experimentator biases, mutations creating new TFBS, TRs having rare secondary roles, etc. Such problems are difficult to correct a posteriori. In some cases, human manual curation is possible to label artifacts that can be subsequently used to train supervised machine learning models, some of which also leverage deep learning and combinations between series  [14]. But this is seldom available. There is currently no systematic detection method or database of known false-positive regions, besides the ENCODE blacklist  [15].
Here, we seek to work at a higher scale and use correlations between processed data. As supervised curation is unavailable, the question is whether we could use another property of the data to identify inconsistent elements. Since TRs most often act in combination through complexe formation  [6] , it follows that biologically significant CREs would likely be clusters of TRs. Similarly, different experimental series should have patterns of corroboration (e. g. where series A says there is a peak, series B says so as well). Here, we define a dataset as one ChIP-seq experimental series for a given TR (in a given cell line, those can be technical replicates or different experiments). For example, RAD21 is significantly associated with CTCF in insulator regions  [17]. As such, RAD21 and CTCF form a correlation group and finding CTCF alone would be suspicious.
Such combinatorics are indeed considered of major interest to CREs detection  [18]  and meta-analysis of datasets is emerging  [19]. As such, we turn to the more general problem of anomaly detection, meaning to identify elements that do not conform to the expected normal patterns  [20] , as a substitute for curation. In this study we focus on detecting anomalous or atypical peaks, where atypical is defined as not respecting the typical TR and/or dataset combinations learned from the data. Removing anomalous peaks will, in turn, fulfill our objective of improving CRE quality. As CREs have high peak density, anomaly detection methods can be used. We consider the ReMap  [21]  database, whose initial curation and uniformized reprocessing workflow provide sufficient quantity and quality to use the outlier detection approaches we propose.
Given the volume of data and the potential complexity of the combinations, Deep Neural Networks (DNN) models are a natural solution and have been used before on genomic data  [22]. They are able to learn complex distributions, not achieved by methods such as PCA, by using multiple layers of increasingly-abstracted representations. Specifically, autoencoders are known to be effective in unsupervised anomaly detection. They also allow us to work at the level of individual loci. Furthemore, this is also a multiview problem  [23]  as each TR can be thought of as one view composed of several datasets. Thus, our approach integrates correlations between datasets and/or TRs, leveraging another strength of NNs. To remove atypical peaks, we propose atyPeak, a stacked convolutional autoencoder, and supply processed data files for selected TFs and cell lines from ReMap.
Representation and processing of cis regulatory modules To apply our method, each CRM is first converted to a 3D tensor representation of the peaks it contains, where the X, Y, Z axes represent respectively genomic position, datasets of origin, and TR of interest (Fig. 1a ). We then use a convolutional autoencoder to perform a lossy compression. The representations are viewed by the model through convolutional filters. They focus first on the correlations between datasets and then between TRs, in a stacked multiview approach (Fig. 1b ). This produces an encoded representation of the CRM, passed to a decoder attempting to reconstruct the original. In the end, each peak is given an anomaly score corresponding to its difference in value in the original and rebuilt representations. Our approach can be applied to any type of sets-of-intervals data from multiple sources in the same format, not only omics.
Validation using artificially generated data Without gold-standard data, and in the absence of precedent readily available methods that are comparable to our approach (cf. Introduction and Methods), we generate artificial data designed to approximate real CRMs to confirm the model's ability to correctly discover correlation groups of sources (a peak's source is its {TR, dataset} pair). Our goal is to simulate biological complexes of TRs, each one being a correlation group. To generate artificial regions, we stack a random number of peaks around a given position. The sources of those peaks belong to one of two (or more) predefined sets of sources representing correlation groups (Additional file 1: Fig. S1 ). The choice of set is made once per CRM. As a result, peaks from each set will significantly correlate only with other members of the same set, forming a correlation group.
We also add to the CRM random noise representing atypical peaks not respecting existing correlation groups. We tested our model's ability to learn which predefined correlation group each peak belongs to, as opposed to the individual peak itself. Indeed, when peaks from a given correlation group are present, the model rebuilds the entire group in the neighborhood of the peaks and not the individual peaks (Fig. 2a , 2 groups). A peak's value in the rebuilding depends on how many sources from its group are present. The more complete the group is, the higher the value (although fully complete groups are unlikely to occur in the actual data). Peaks added to the rebuilt tensor by the model are called phantoms (Fig. 2 ). Biologically, this means that the model will identify common TRs and/or dataset combinations.
Stability to different group characteristics Our model is still effective with more complex and realistic artificial data models. For instance, the number of binding sites across TRs differ but we show that the model is not biased by differences in abundances between correlation groups (Additional file 3: Fig. S3a ). Furthermore, biological correlation groups are not mutually exclusive (e. g. , for the TRs A, B and C, there can be both an "ABC" and an "AB" group) and the model Fig. 1  a Workflow and model description. Once candidate regions (ReMap-identified CRMs) are set, we build tensors of peak presence representing them. The X axis represents the position along the genome, while the Y and Z axis are dataset and TR identifiers respectively. The tensor has a value of 1 is a peak for this TF in this dataset (i.
e. , for this source) is present, 0 otherwise. The atyPeak model will lossily compress this representation. This will result in losing anomalies and other finer details, by learning correlation groups for the rebuilding instead of individual peaks. At the end, each peak is given an anomaly score corresponding to the mean autoencoder reconstruction error, the difference between the original (grey) and rebuilt (red) representation. Scores are then added to the original BED file. Full source code and documentation are available at < https:// github. com/ qferre/ atype ak >. b Model structure. During the encoding, the CRM are viewed by the model through convolutional filters to focus on the correlations between datasets and then between TRs. We use two type of filters (combinations of datasets, then combinations of TRs) successively in a stacked multiview approach.
Scalability and the information budget Any compression is characterized by its aggressiveness: one that is not aggressive enough can afford to learn details (for us, the noise we want to remove) but a too aggressive compression might lose too much information. In our method, this depends on the relative information budget, which is the ratio between the data's information and the model's entropic capacity. Biologically, this means the budget to be used depends on the size of the database: the number of sources considered and the number of relevant correlation groups in the data. For example, in Fig. 2b  the Fig. 2  The atyPeak model learns correlation groups. In each case, the tensor at top is the original representation and the bottom one is what is rebuilt by the model. The model was trained on artificial data.
Application to real biological data Having evaluated our model on artificial data, we processed biological data from the ReMap project. We used selected CRMs for the Jurkat, Hela, K562, MCF7, CD34 and ESC cell lines. This data was sparser than the artificial datasets, as not every dataset contained every TR. This necessitated "crumbing" (Additional file 8: Fig. S9 ) and adapting the information budget. Parameters are detailed in Additional file 9: Table  S2  and were chosen thanks to the Q-score. We covered a variety of cell line profiles, ranging from Jurkat's sparse genomic binding data with many datasets concerning only one TR, to high-dimensionality examples such as K562 proving our model's scalability potential. Figure  4  shows two representative examples of CRMs in HeLa along with their rebuilding by the model.
They could also be high-noise regions, although this is unlikely as they were among the richest CRMs in 2018. Supervised data would be required to determine the correct hypothesis. At the individual peak level however, we consider the update ratio for peaks of the same TR within the same CRM, which would confirm the binding of this TR here by drawing The annotated BED data files with the corresponding atyPeak scores are available at <https:// github. com/ qferre/ atype ak-files> or as a UCSC browser session at <http:// genome-euro. ucsc. edu/s/ qferre/ atyPe ak_ hg38>. We can see on this figure an example of rich CRE with many peaks, and a poorer CRE where many correlators for those TRs are missing which predictably has a lower score. As detailed previously, our approach estimates how "typical" each peak is, with respect to the usual combinations between sources (TRs and/or datasets) for a given cell line.