dataesr/scientific-paragraphs-categorization huggingface dataset:
Binary segmentation of volumetric images of porous media is a crucial step towards gaining a deeper understanding of the factors governing biogeochemical processes at minute scales. Contemporary work primarily revolves around primitive techniques based on global or local adaptive thresholding that have known common drawbacks in image segmentation. Moreover, absence of a unified benchmark prohibits quantitative evaluation, which further clouds the impact of existing methodologies. In this study, we tackle the issue on both fronts. Firstly, by drawing parallels with natural image segmentation, we propose a novel, and automatic segmentation technique, 3D Quantum Cuts (QCuts-3D) grounded on a state-of-the-art spectral clustering technique. Secondly, we curate and present a publicly available dataset of 68 multiphase volumetric images of porous media with diverse solid geometries, along with voxel-wise ground truth annotations for each constituting phase.
One can observe that this task is similar to the binary segmentation of porous media representations in the key sense that both are concerned with separating out a single region of interest (salient-object/solids) from the backdrop (background/pore-space) in a one-vs-rest fashion. Moreover, the region of interest in both cases is texturally homogenous and unique from the immediate spatial surroundings. Based on this, we argue that salient object detection methods are potentially more suitable for application to porous media images as opposed to classical thresholding techniques. It is worth noting that, quite 3 unlike the segmentation of porous media images, the field of salient object detection is ripe with numerous novel contributions, both in terms of diversity of segmentation techniques and number of publicly available benchmark datasets.
In an extensive exploratory work undertaken by  [16] , 29 methods were evaluated on 7 benchmark datasets across multiple evaluation metrics. Among the tested methods, Quantum Cuts (QCuts)  [17] [18] [19] , a recent segmentation technique based on Quantum Mechanical principles, consistently stood out as the best performing method. Albeit unsupervised, QCuts provided results on par with the supervised techniques, and even outperformed them over some datasets. In light of these findings, we hypothesize that an automatic, unsupervised and class-independent salient object detection method like QCuts lends itself to application for segmenting volumetric tomography images of porous media. Furthermore, in addition to the lack of efficient segmentation systems, another critical factor hindering progress in the field of porous media images' segmentation is the absence of a benchmark dataset with annotated ground truth that can be used to qualitatively and quantitatively evaluate 3D porous segmentation methods.
Curation of such datasets is made harder bacause of the fact that, for the case of naturally occurring porous media such as real soils, obtaining a voxel-wise ground truth is not possible  [7, 13]. Morphological characteristics such as the porosity (percentage of space occupied by pores) of a real sample and that calculated from a segmented image are often compared to gauge the segmentation accuracy. However, such measures are not adequate as the total porosity of a soil sample is affected by pores that might not be visible at the studied X-ray resolution  [2, 7]. As a workaround, researchers have resorted either to generating synthetic soil images of known porosity by working backward from binary images  [7] , or to using artificial systems whose porosities can be manually controlled  [5].
Moreover, the porosity measure itself is not descriptive enough as it only takes into account the overall percentage of pores, not their spatial location. This absence of benchmark datasets renders the quantifiable comparison among the segmentation results impossible. Collectively, the evidence presented above highlights key issues concerning both the outdation of segmentation techniques and the uncertainity regarding their performance that arises from the absence of annotated datasets. In this study we address these two issues and make the following novel contributions: i) QCuts-3D: an extension by 3D reformulation of the state-of-the-art 2D salient object detection method of QCuts, tailored exclusively for segmenting volumetric tomography images of porous media, ii) SIMUPOR dataset: a collection of high-resolution multiphase volumetric images along with expert annotated voxel-wise ground truth for each phase.
This is one of the key reason behind the obsolence of thresholding-based techniques since the earlier image segmentation efforts. One of the most widely studied natural 2D image segmentation problems over the recent past is the task of salient object detection, which deals with identifying a single region of interest that is visually appealing and salient from the spatial surroudings. Quite similar to binary segmentation of porous media images, salient object detection aims at a one-vs-rest categorization of the image elements into the foreground (region-of-interest) and the background. A wide variety of salient object segmentation literature exists, along with numerous labelled datasets for benchmarking. In a thorough quantitive evaluation of 29 salient object detection over 7 different datasets, the methods of DRFI  [29] , RBD  [30] , ST  [31] , DSR  [32] , MC  [33]  and QCUT  [19] , came out as the top performing methods;  [19]  being the best among unsupervised methods.
Unary Potentials in QCuts-3D In  (10) , the unary term () encodes prior information about the labeling of nodes and is related to the potential of a node to belong to the background. In native QCuts, superpixels occupying the boundaries of the 2D image are assumed to belong to the background, based on the observation that objects of interest in natural images frequently tend to lie close to the center of images  [16, 49]. Therefore, () is set to a very high value for these nodes and zero for all the others. However, such a location-based prior is not suitable for the case of images of porous media. This is demonstrated in Figure  4  which compares a pixel-wise dataset-wide average of ground truth annotations of 3 popular salient object detection datasets and 2D slices of the SIMUPOR dataset.
MSRA10K PASCAL-S SED2 SIMUPOR Figure  4  Mean ground truth annotations for  [50] ,  [51] ,  [52]  (reprinted from  [16] ) and SIMUPOR dataset. Earlier, for such volumetric images,  [53]  proposed to use a manual selection of seeds for each phase by a skilled operator. However, the authors themselves identified a need for an automated process, as manual selection makes the overall process expensive in terms of processing time and also influences the segmentation results based on the operator's subjectivity. Moreover, owing to limitations in visualization, manual selection of seeds is not practical in the case of 3D images  [54]. In light of the aforementioned facts, in QCuts-3D, the only assumption we make about the pore space is that it occupies the lower end of the grayscale intensity distribution.
The Benchmark Dataset: SIMUPOR In this study, a benchmark dataset (aka SIMUPOR) of 68 3D volumetric images of porous media with varying grain geometry and composition is composed 1. 3D volumetric images were obtained from the experiments conducted in  [56]  to study the effect of grain geometry on the morphology of non-aqueous phase liquids in porous media. The volumes correspond to samples from 34 different experiments, each corresponding to a specific constitution of porous medium. Among the 68 volumetric images used, 40 belong to the experiments that employ silica sand to model the porous media whereas the remaining 28 used quartz crystals. In addition to this variation in the shape of grains, there is also a variety in the size, with the median grain diameter ranging from 0. 179 to 0. 433 mm.
Performance Evaluations Table  1  and Table  2  chronicle the results of our experiments over the SIMUPOR dataset by using the aforementioned evaluation metrics. Table  2  presents the performance of the state-of-the-art automatic segmentation method of  [13]  and QCuts-3D on images categorized with respect to different grain shapes. On the average, QCuts-3D achieves a boost of 26% in AUROC, 23. 5% in IoU and 68. 9% in ME. Similarly, Table  2  shows the same performance criteria evaluated across a variety of grain sizes. The proposed method achieves a boost of 27% in AUROC, 23% in IoU and a 70% decrease in ME, on the average. Furthermore, we also observe a decrease of 28. 2%, 16. 1% and 28. 9% in the standard deviation of AUROC, IoU and ME, respectively. The results indicate a significantly superior performance in terms of accuracy and consistency.
The latter is particularly important as QCuts-3D is more robust to the variations in the grain size as compared to the state-of-the-art method  [13]. 11 Table  1  Performance of QCuts-3D and the competing method  [13]  across different grain sizes. In terms of qualitative comparison, Figure  7  presents the results of  [13]  and QCuts-3D on the images from the SIMUPOR dataset. One can clearly observe that, owing to the presence of multiple phases in the images, the adaptive thresholding technique of  [13]  fails to provide adequate results, which is not surprising at all, given the fact that this method assumes a priori the existence of only two distinct populations of voxels in an image. The method fares well in identifying the gas as the pore phase, but falls short when it comes to liquid or oil phases, which have intensities closer to the solid phase.
Computational Complexity Analysis MATLAB implementation of QCuts-3D and  [13]  were tested in MATLAB R2018a on an Intel Xeon CPU E5-2690 processor running at 2. 60 GHz with 64 GB of memory. Computational efficiency for both methods was measured by applying them over the entire SIMUPOR dataset and averaging the overall run-times. We observe that, on the average, the competing method  [13]  took 937. 85 minutes on a single volumetric image of size 256x256x256 whereas the proposed multi-resolution scheme took only 100. 96 seconds. This presents around 500fold less computational time due to the fact that the competing method involves multiple passes over the entire image as part of its search for optimal window size, which has a substantial increase on the computational complexity. In contrast, QCuts-3D is devised from the ground up to consist of efficient end-to-end graph-cut operations and is kept free from any need to repeatedly revisit the image space for parameter optimization.
Conclusions and Future Work In this work, we proposed a novel, automatic and unsupervised 3D segmentation technique for digital volumetric images of porous media. We also presented a benchmark dataset consisting of 68 such volumetric images with corresponding voxel-wise segmentation ground truths for multiple phases. We performed an extensive comparative evaluation over this dataset of the proposed QCuts-3D against the state-of-the-art segmentation method  [13]. Both quantitave and qualitative results revealed that QCuts-3D achieves a significant improvement in segmentation accuracy and in computational efficiency. Moreover, the proposed technique was also found to be more robust to variations in compositional elements of the pores and the geometry of grains composing the porous media. Visual comparison further revealed the fact that the regional boundaries are better preserved using the proposed supervoxel-based approach, as compared to localized thresholding in cubical windows employed by the competing method.
Finally, the graphical representation of the volumetric image lends itself to an extension towards multiphase segmentation in a supervised approach. Future work involves investigating ways of improving the computational efficiency of the proposed method and extending it towards application in multiphase segmentation. Currently, the primary computational bottleneck in the proposed pipeline is the multiresolution supervoxel-based oversegmentation. We aim to harness the power of GPUs to expedite and parallelize this step. Furthermore, we will also explore the possibility of devising new single-pass multiphase segmentation approaches by utilizing volumetric stacks of images acquired using scanning at different energy levels. Moreover, the introduction of the first benchmark (SIMUPOR) dataset can potentially usher in an era of rapid influx of modern segmentation techniques such as Deep Learning-based supervised approaches, that can learn more efficient graphical representations  [66]  to segment the images using the provided ground truth annotations.
Motivation: We present dnadna, a flexible python-based software for deep learning inference in population genetics. It is task-agnostic and aims at facilitating the development, reproducibility, dissemination, and reusability of neural networks designed for population genetic data. Results: dnadna defines multiple user-friendly workflows. First, users can implement new architectures and tasks, while benefiting from dnadna utility functions, training procedure and test environment, which saves time and decreases the likelihood of bugs. Second, the implemented networks can be re-optimized based on user-specified training sets and/or tasks. Newly implemented architectures and pretrained networks are easily shareable with the community for further benchmarking or other applications. Finally, users can apply pretrained networks in order to predict evolutionary history from alternative real or simulated genetic datasets, without requiring extensive knowledge in deep learning or coding in general.
, 2021; Burger et al. , 2022; Meisner and Albrechtsen, 2022; Perez et al. , 2022; Qin et al. , 2022). Previous studies have made their implementations available at least for reproducibility and sometimes with a specific effort for re-usability. Even so, each of them focuses on a specific network for a specific task. Adapting them requires careful understanding of the code and its direct modification since many options are hardcoded. This is not only error-prone, but also leads to a rapid code divergence between parallel projects, accompanied by complex maintenance. The community demands flexible and rigorous tools as demonstrated by stdpopsim, a library for population genetic simulations which allows contributions from many researchers  (Adrion et al. , 2020a). In genomics a suite of tools has been developed to facilitate deep learning applications (e.
Software dnadna is a python-based software for population genetics inference that enables researchers to (1) develop new networks or re-use existing architectures, (2) train them for a given task (regression, classification, or a mix of those), and (3) share them in such a way that users can easily apply these trained networks to their own dataset. In particular, it already implements several neural networks that have been tested for inferring demographic and adaptive histories from genetic data. Pre-trained networks can be used directly on real and simulated genetic data for the prediction step. Networks can also be re-trained on new simulations (e. g. corresponding to another species or evolutionary model) and/or to solve other tasks (e. g. classifying introgressed vs non-introgressed segments or inferring recombination).
Finally, any user can implement new architectures and tasks, while benefiting from training procedure, test environment, routines that may be otherwise overlooked (such as proper preprocessing or efficient data loading), and the possibility to easily share a network to facilitate re-use and benchmarking. Figure  1  provides an overview of dnadna steps. It is accessible without coding knowledge thanks to its YAML configuration files which provide the user with a variety of options at each step of the process. Because each use-case has its own specificity, we developed a system which allows users to implement plugins (e. g. data transformations, simulators, or networks) without modifying the core of the code. dnadna is a Python (≥3. 7) package with multiple dependencies, the main one being the open source machine learning library PyTorch.
It has a command line interface and an API (mlgenetics. gitlab. io/dnadna/api. html). It is highly flexible thanks to a structured configuration file system based on YAML and JSON Schema. dnadna is dual-licensed under the GNU Lesser General Public License (LGPLv3+) and the compatible CeCILL-C Free Software License Agreement. Release 1. 0 is available from PyPI at pypi. org/project/dnadna/ and Anaconda at anaconda. org/mlgenetics/dnadna. Docker images are available at hub. docker. com/u/mlgenetics.
Tutorial examples We showcase various dnadna use-cases via tutorials that will continue to expand. It is not required to have coding knowledge to perform similar A first tutorial walks the user through the complete process from configuring and generating simulated genetic data, to running data pre-processing, and training a convolutional network to solve a regression task (here predicting the parameters of a two-step population size history). A second tutorial solves a classification task instead (Figure  1G ). A third tutorial helps users who already have simulated data, to familiarize themselves with dnadna and train a SPIDNA network on a provided dataset. Finally, we provide tutorials in the form of jupyter notebooks (mlgenetics. gitlab. io/dnadna/tutorials. html and Figure  1F ).
Xue et al. (2021)) ; (2) preprocessing, mainly to filter out examples that do not fit minimal requirements and split the rest into train/validation/test sets; (3) training neural networks; (4) predicting on test or real datasets using optimized neural networks. Note that simulations can be skipped if the user already possesses a labelled dataset. Similarly, training can be skipped if the user reuses a pretrained network. Here is a subset of options at each step: (1a) generating simulations: name of predefined scenario to be simulated and its related parameters, such as number of individuals, number of replicates, mutation rate, demographic parameters;.
Acknowledgements TAU and Kepler GPU platforms. DIM One Health 2017 RPH17094JJP, Human Frontier Science Project  RGY0075/2019 , Paris-Saclay CDS 2. 0 (IRS), CNRS-80Prime  TransIA ,  ANR-20-CE45-0010-01  RoDAPoG for funding. M Fumagalli  and the participants of the School " Inference using full genome data " (DFG  SPP1819 ) for beta-testing. S Ribeiro  for her comments.
Xue et al. (2021)) ; (2) preprocessing, mainly to filter out examples that do not fit minimal requirements and split the rest into train/validation/test sets; (3) training neural networks; (4) predicting on test or real datasets using optimized neural networks. Note that simulations can be skipped if the user already possesses a labelled dataset. Similarly, training can be skipped if the user reuses a pretrained network. Here is a subset of options at each step: (1a) generating simulations: name of predefined scenario to be simulated and its related parameters, such as number of individuals, number of replicates, mutation rate, demographic parameters; dnadna (1b) handling simulations: location on disk and filesystem layout; (2) preprocessing: initial data transformations, filtering values such as minimal number of sampled individuals or SNPs; (3a) architecture design: network name and related arguments (number of filters, layers,.
The estimates for a "complete" genome are given by the averaged predicted values. The error corresponds to the relative squared errors averaged over the whole time period. G: Illustration of a classification task, following Quickstart tutorial 2. The network was trained on a toy dataset (2000 independent population samples split into training and validation) and tested on 2000 additional samples to discriminate whether the population underwent a decline or an expansion of its size. The contingency table shows that for this classification task, the accuracy is 85. 3% on a test set. See mlgenetics. gitlab. io/dnadna/tutorials. html for details on both experiments.
Overview of INTEL-IRRIS project As indicated previously, INTEL-IRRIS has the ambition to make digital and smart farming technologies attractive & more accessible to SHFs. Therefore, a trade-off will be made to meet the main objective which is to provide a low-cost irrigation system that can be deployed by SHFs in an out-of-the-box manner. While the soil moisture sensing part adopts a simple yet robust and efficient low-cost design largely inspired by many do-it-yourself initiatives and previous contributions  [5, 6, 7]. Unlike those low cost sensors that too often provide nonreliable data, INTEL-IRRIS will contribute in greatly improving the quality of the collected data with (i) advanced calibration of the different sensors and (ii) allowing more plant and soil parameters to be pre-configured into the control system.
The low-cost soil moisture sensing part The low-cost soil moisture sensing part is based on an DIY IoT hardware platform that has been developed and validated in 2 EU projects (EU H2020 WAZIUP and EU H2020 WAZIHUB) with piloting sites deployed in several sub-Saharan countries. The microcontroller is the Arduino ProMini in its 3. 3V and 8MHz version. Software building blocks include long-range data transmission with LoRa radio technologies (possibly in 868MHz, 433MHz and 2. 4GHz bands; and using Lo-RaWAN specification) and advanced power management that allows the device to be fully autonomous for more than 2 years when running on 2 AA batteries and transmitting data once every 60 mins  [12]. The hardware parts are illustrated in Figure  1. The PCB are available as open-source and technology adaptation to local market availability will be realized by UORAN1 Algerian and ENSA Safi Moroccan partners.
The edge-enabled IoT gateway approach The INTEL-IRRIS edge-enabled IoT gateway is based on the WaziGate framework that has been developed over the years by WAZIUP in the context of several international research development projects. WaziGate itself took its root from the early "Low-cost LoRa IoT gateway" framework  [12]  developed by University of Pau since 2015 for building open and versatile LoRa IoT gateway with a Rasp-berryPi board. WaziGate is now a separate development line with maturity level of TRL8-9  [14]. The WaziGate framework is highly suitable for implementing the edge-enabled IoT gateway approach as customized applications can be hosted in the gateway and use the gateway's innovative features for edge approach such as embedded database, embedded web-based data visualization module, software container management and cloud synchronization features as well as remote management if this is desirable.
Architecture WaziGate software framework uses a microservice architecture to manage all local applications (referred to as WaziApps). This architecture allows each WaziApp to operate independently and in isolation using Docker containers. WaziApps can be written in various programming languages and they communicate through a set of restful APIs. Actually, even WaziGate's system components are considered as independant WaziApps interacting with each other through APIs. Figure  4  shows the component diagram of the WaziGate software architecture. The top level is the user domain, the middle level represents the WaziGate software and the bottom level is the hardware, i. e. the RaspberryPi. On top of the RaspberryPi OS is a Docker Engine, which handles all necessary images/containers. All WaziGate's components expose a well-defined API enabling efficient and secured calls to all WaziGate's services by user-defined WaziApps.