dataesr/scientific-paragraphs-categorization huggingface dataset:
Pan-Cancer Optimal Panel Length Comparisons Panel length to reach full informativity was compared across all cancer types using the PIO length analysis module. For each cancer, an optimal panel was defined using the four different approaches proposed by PIO. These approaches were first compared using the overall proportion (percentage) of patients diagnosed according to panel size in kb smoothed using a general additive model. A focus was then proposed for each metric, comparing the panel size differences, allowing us to reach 100% informativity between PIO and classic approach. This optimal panel size was then extracted for each cancer type and used for exploratory correlations with the total number of patients and different markers of DNA alterations extracted from Thorsson et al. 83.
Examples of Application For panel size optimization, a 293-gene (802-exon) panel targeting genes recurrently mutated in hematological malignancies  4  was shrunk using the PIO custom mode with UP or UPKB metric and mutations grouped by exon. The performance of the resulting 140-exon panel was then evaluated in 928 DLBCL patients who benefited from the full panel. 3 r the evaluation of our local routine 110-gene panel, PIO was first used in custom mode (mutations grouped by gene, UP metric, and at least one patient per mutation) to evaluate the predicted performance in the preloaded data set (predicted). The proposed optimized 33-gene panel was then evaluated in test mode on our local cohort (observed). For panel benchmarking in AML, five commercial myeloid panels  4  together with our local panel were evaluated using PIO in panel test mode (mutations grouped by gene, UP metric, and at least one patient per mutation) on the preloaded AML data sets in the PIO.
A graphical output illustrates the frequency and nature of the most frequently mutated genomic intervals in the overall cohort, and the data are summarized in a downloadable table. A second graphical output presents a co-occurrence plot of mutations in the selected genomic intervals at the individual level. Although it is not the main purpose of PIO, additional information about potential mutation actionability extracted from the Clinical Interpretation of Variants in Cancer (CIVID) database (https:// civicdb. org/home, last accessed July 12, 2021) is also provided in a separate tab. In addition, the length module compares four different approaches to optimize panel design to maximize the informativity/size ratio (see Materials and Methods).
Examples of Application The performance of PIO was assessed in different settings. The size of a 293-gene (802-exon) panel used in DLBCL  3  was first optimized while retaining maximum informativity of at least one mutation per patient. Using the PIO custom mode with UP metric and mutations grouped by exon, we predicted on the preloaded DLBCL data sets that 93. 8% of DLBCL patients could be diagnosed with a subset of only 140 exons from this panel (Figure  5A ). This prediction was compared with sequencing results of a public data set of 928 patients with DLBCL who were analyzed by the full panel. 3 n this cohort, 95. 8% of patients presented at least one mutation in one of these 140 exons (Figure  5A ). Using PIO with either UP or UPKB metric allowed reduction of the panel size from approximately 300 kb to <100 kb whilst keeping the same level of informativity (Figure  5B ).
Discussion We propose herein PIO, an open-source software to help establish NGS panels and which offers several applications, such as mutation exploration, panel length optimization, and commercial panel benchmarking. Newman et al  84  have already proposed a method to optimize panel informativity in the design phase of cancer Figure  3  Optimal panel length comparison according to classic and Panel Informativity Optimizer (PIO) approaches. A: Overall pan-cancer informativity using the classic or PIO approach without comutation optimization, with either number of unique patients (UP) or number of UP per kilobase (UPKB) metric. Overall informativity is represented using a general additive model independently for each individual cancer. B: Differences between PIO and classic approaches for UPKB metric.
Few other tools exist to select an optimal set of genomic intervals, which are usually optimized for specific situations and do not offer the same level of flexibility as PIO. For example, Precision Trial Drawer 85 is a software specifically designed to estimate the number of patients needed to be screened for precision oncology studies. To our knowledge, the tool that comes closest to PIO is OncoPAD. 86 Although accessible and easy to use, OncoPAD's main algorithm is restricted to a set of known driver genes (n Z 576), using intervals defined at both the gene and the exon level. A custom set of genes can be used but is limited to 1000. Besides, OncoPAD does not allow users to set up the informativity level, and there is no mutation exploration module. OncoPAD is thus designed for having a quick insight into mutation panels restricted to the main driver mutations with potentially actionable targets.
Considering potential cohort bias is also important when using PIO. Although a low variance in the number of genes needed for full informativity was observed for most cancers, some were characterized by high heterogeneity between cohorts. This heterogeneity could be explained by disease heterogeneity (several molecular subtypes) or differences in patient inclusion criteria. To circumvent this, PIO allows us to interrogate any custom data set, hence enabling the user to select representative studies according to its specific clinical settings. Although allowing us to maximize informativity, PIO does not account for specific settings, such as the prognosis or theragnostic value of a given mutation. Hence, although clinically relevant events will be included according to their relative frequency and informativity, the selection of a set of genes by PIO does not preclude the manual inspection of the panel by a trained molecular biologist to verify and add clinically relevant and mandatory mutations.
Both land-cover classification and building extraction are evaluated using multispectral images from two publicly well-known datasets: Zurich and Vaihingen. This paper is organized as follows. We first review related works on the use of semantic prior knowledge in Section II. Then, we present some basic definitions of graphs, hierarchical watersheds and attribute profiles in Section III. In Section IV, we introduce the proposed Watershed-APs and our strategies to integrate semantic knowledge within its construction. Finally, experiments conducted on the two aforementioned remote sensing datasets are given in Section V with extensive analysis and discussion in Section VI. Conclusions are drawn in Section VII.
And, in  [22] , crop classification in SAR time series are performed using Auto Encoders (AE), Convolutional Neural Networks (CNN) and FCN, and then post-processed using prior knowledge regarding crop dynamics, i. e. expert's knowledge about which crop transitions might or not occur over time in the same field. Though deep learning models perform well in computer vision tasks in general, including remote sensing imagery, there are advantages of using feature extraction methods based on hierarchical representations and morphological operators, such as the proposed Watershed-APs. In particular, we can mention the interpretability of the method when compared to the black box parameters of deep learning models, and the low need for lots of ground-truth data, which makes those morphological methods well adapted to datasets with scarce annotated samples.
B. Watershed-APs with semantic prior knowledge As discussed in Section II, user-defined markers and prior knowledge from training pixels are valuable tools for optimizing the construction of various image representations, such as hierarchical segmentations, as well as for post-processing such representations. In this article, we investigate the use of prior knowledge in the construction of Watershed-APs. We present a general framework for including prior knowledge at different stages of the Watershed-APs construction, followed by two instances of this framework that are later validated on multi-spectral remote sensing datasets. AP and its variants are essentially unsupervised feature extraction methods, in which only the spectral values and the relative position between pixels are taken into consideration.
Due to the interclass similarity and intra-class variability of pixel in remote sensing images, hierarchical segmentations based only on pixel values often do not reflect the inner semantic structure of the data. With that in mind, we rely on training pixels to obtain hierarchical watersheds whose regions partially reflect the semantic structure of the input data. In practice, we aim to enforce regional minima at the regions with high probability of belonging to any given ground-truth class. This is done through a combination of the methods proposed in  [12] ,  [13] ,  [10] , as described below. Given a dataset I (e. g. a panchromatic or an RGB image) and its training set composed of n classes, we compute its hierarchical watershed using prior knowledge as follows: (i) Training of a classifier using the training set of I and computation of per-pixel class probabilities (p 1 ,.
, p n ) for all pixels of I; (ii) Combination the class probabilities into a single map µ; (iii) Computation of a weighted graph G P = (V, E, w P ) from µ; (iv) Computation of a weighted graph G G = (V, E, w G ) which represents a gradient of I, i. e. , edge weights indicate the dissimilarity between neighbouring pixels; (v) Combination of the weight maps w P and w G into a map w GP ; (vi) Computation of the hierarchical watershed of G GP = (V, E, w GP ) for a given sequence of minima of G. Readers may note that each of the above steps can be implemented in many different ways. In our experiments, our choices have been adopted based on solutions present in the literature and on empirical results on the tested datasets. In the first step of our method, we are aware that: (1) there might be sample pixels of a given class whose spectral values are not represented in the training set, and (2) there might be pixels in the training set with very similar spectral signatures but which belong to distinct classes.
In other words, the final results will be similar to the ones obtained with the original pixel values. In the step (iii), a weighted graph (V, E, w P ) is obtained from the combined probability map µ. In our experiments, we chose to compute edge weights as the maximum between the probability values of neighbouring pixels based on a few experiments with the datasets described in the conference version of this article  [9]. In the steps (iv) and (v), a gradient (V, E, w G ) of I is computed and then combined with (V, E, w P ). In our experiments, this combination will be simply implemented as a multiplication of edge weights, similarly to  [10]. We note that the proposed method is related to ones introduced in  [19] ,  [20] , the main difference being the type of hierarchy under consideration and how the original data is combined with the prior knowledge.
For instance, if C is a Random Forest composed of m trees, obtained from t training samples with f features, the time complexity to build such a forest is O(m×t×log(t)×f ). Taking into account that f is a small constant in our experiments, the time complexity to build such a forest is O(m × t × log(t)). In An illustration of the proposed method is given in Fig. 4. A crop of the image zh20 from the Zurich dataset  [30] , its ground-truth composed of six semantic classes and a class probability map (obtained as described in Algorithm 1) are shown in Fig. 4  both results, we observe that images in (c) preserve some of the boundaries between distinct semantic classes, as, for instance, between the regions belonging to classes trees (represented in dark green in the ground-truth) and grass (light green) in the lower right corner of the reconstructed images.
Therefore, we can conclude the proposed methods for including semantic prior knowledge into Watershed-APs present equivalent time complexities. Fig. 6  illustrates the method described in Algorithm 2 on a cropped patch of an image from the Vaihingen dataset  [31]. The image and its ground truth regions are given in Fig. 6 (a). In Fig. 6 (b) and (c), we give thresholded versions of the probability map for the class trees (represented in green in the ground truth), along with the image reconstructions obtained from a hierarchical watershed filtered using each of those probability maps, as described in Algorithm 2. Similarly, Fig. 6(d ) and (e) show the results obtained for the class cars (represented in yellow in the ground-truth image). For a better visualization, all image reconstructions are represented in false colors.
V. EXPERIMENTAL SETUP In this section, we conduct experiments to evaluate the performance of the proposed Watershed-AP (computed with and without prior knowledge) in the context of land-cover classification and building extraction of remote sensing images. We first describe the multispectral images considered in our study, as well as the experimental settings used for evaluation. We provide detailed analysis and show that oftentimes Watershed-APs outperform AP and its variants including SDAP  [32] , α-AP  [25]  and ω-AP  [25] , on both datasets in the next section. Experiments were performed in Python 1  using the open source Simple Attribute Profile (SAP)  2  and Higra  [33]  libraries  3.
A. Datasets Our experiments were conducted on the multispectral remote sensing Vaihingen [31] and Zurich  [30]  datasets. The Vaihingen dataset [31] contains 38 images collected over the city of Vaihingen, Germany. Each image has a spatial resolution of 9 cm and is composed of four channels (near infrared, red and green and blue) plus a digital surface model (DSM). As done in  [34] , we only consider the first three channels, namely near infrared, red and green in our experiments. Images width and height range in the intervals [1388, 3816] and [1281, 3313], respectively. Ground-truth annotations, which are provided for each of the 38 images, consist of at most six thematic classes: impervious surfaces, buildings, low vegetation, trees, cars, and background/clutter. Following previous works  [34] , we only consider the 16 images for which ground-truth annotations were provided during the ISPRS semantic labeling contest.
Moreover, the background class is not used for land-cover classification. To perform building extraction, the ground-truth labels are divided into two classes: buildings and background (i. e. the union of all remaining semantic classes). Training samples were extracted from eleven images (image IDs: 1, 3, 5, 7,  13, 17, 21, 23, 26, 32, 37) , and the remaining five images (image IDs  : 11, 15, 28, 30, 34)  were used for evaluation. For each semantic class, training samples were composed of 1% of randomly selected pixels in the training images, leading to the number of training and test pixels per semantic class given in Table  I. The test images of the Vaihingen dataset and their ground-truths are shown in Fig. 7. The Zurich Summer dataset  [30]  is a collection of 20 images obtained from a QuickBird acquisition of the city of Zurich, Switzerland, in August 2002.
The images in this dataset have various dimensions, with widths and heights in the [622, 1639] and [782, 1830] ranges, respectively, and are composed of four channels (near infrared, red, green and blue). The ground-truth provided for each image consists of at most nine thematic classes, namely: roads, buildings, trees, grass, bare soil, water, railways, swimming pools and background. For the building extraction task, all classes except buildings are merged into a single background class. Following previous works  [35] ,  [34] , training pixels were extracted from the first fifteen images of this dataset, and the remaining five images (zh16, zh17, zh18, zh19 and zh20) were used for evaluation. As for the Vaihingen dataset, the background class of the Zurich dataset is not considered for land-cover classification.
B. Experimental settings Most works in the literature evaluate AP and its variants in the following way: training and test pixels come from the same remote sensing image, which allows training and test features to be extracted from the same hierarchical representation of the image under study. As discussed in  [5] , this setting does not generalize well to more realistic scenarios, in which we may have a dataset composed of several images without any annotation. For that reason, we extend the evaluation on the Zurich dataset already presented in our conference paper  [9]. In the present paper, training and test features are extracted from independent hierarchical representations computed from the training and test images. AP and its variants, including Watershed-APs, were computed with the usual area and moment of inertia (MoI) attributes.
The following ten area thresholds and four MoI thresholds were adopted for both datasets: λ area = {25, 100, 500, 1000, 5000, 10000, 20000, 50000, 100000, 150000} λ moi = {0. 2, 0. 3, 0. 4, 0. 5} The only exceptions are the Watershed-APs filtered using semantic prior knowledge, as described in IV-B2. To compute those Watershed-APs, the hard-coded criteria based on the area and MoI attributes are replaced by the nodes' probability of belonging to each ground-truth semantic class. In our experiments, the set T of threshold values used by Algorithm 2 is defined as an interval of evenly spaced numbers ranging from the minimum to the maximum values of each classification probability map. In order to obtain a similar number of features as the other APs, different numbers of threshold values were considered for each task and for each dataset (see Table  III ).
A. Land-cover classification Tables IV-IX present the results of land-cover classification on the Vaihingen and Zurich datasets. We compare the performance of the following methods: the baseline, in which every pixel is represented by its NIR+RG or NIR+RGB values, AP-maxT and AP-minT obtained by filtering the max-and min-tree, respectively; AP  [3]  obtained as a concatenation of AP-maxT and AP-minT; SDAP  [32] ; α-AP and ω-AP  [25] ; and the Watershed-APs computed/filtered with and without prior knowledge. To simplify the notations, Watershed-AP  computed without prior knowledge is denoted as A-WS-AP, and the Watershed-APs computed and filtered using prior knowledge are denoted as A-CPWS-AP and A-FPWS-AP, respectively, where A is the attribute used in the construction of the hierarchical watersheds, namely Area, Dynamics (Dyn) and Volume (Vol).
On the Vaihingen dataset, as shown in Tables IV and V, the Area-WS-AP and Volume-WS-AP, which are computed without any prior knowledge, outperform all other methods in the literature in terms of OA and κ scores. The same is true for Area-CPWS-AP and Vol-CPWS-AP, as well for all three FPWS-APs. Our best method, namely Vol-FPWS-AP, outperforms AP by 1. 25% and by 1. 69 in terms of OA and κ scores, respectively. Regarding the use of prior knowledge for WS-APs, all three FPWS-APs performed better than their respective CPWS-APs and WS-AP counterparts. The most meaningful improvements were observed for Dyn-FPWS-AP, which presented OA, AA and κ scores more than 1% higher than Dyn-WS-AP. Considering the classification result per semantic class given in Table  VIII , we observe that the Watershed-APs yielded the best results for the buildings, low vegetation and tree classes, but falls behind AP on the classification of impervious surfaces and cars.
To provide further insight into the performance of APs in the context of semantic segmentation, we compare one of our results with a recent deep learning method  [34]  (see Table  VI ). In  [34] , the authors propose an adaptation of the Fully Convolutional Network (FCN) to consider sparsely annotated training data. They trained their proposed model with different types of scribbled annotations, including points, lines and polygons. On both datasets, their baseline is the FCN trained on the whole set of training pixels, and their best results were achieved by the FCN-Festa-dCRF method trained on scribbled lines. The FCN-Festa-dCFR and FCN methods were trained on 480,593 and 54,373,518 samples, respectively, while that our Vol-FPWS-AP was trained on 543,707 samples. On this dataset, Vol-FPWS-AP approached the scores of FCN-Festa-dCFR on three classes, namely impervious surfaces, buildings and trees, overperformed the later on the low vegetation class, but presented poorer results on the classification of cars.
With respect to FCN, both Vol-FPWS-AP and FCN-Festa-dCFR performed worse in general, except for the trees class, for which Vol-FPWS-AP and FCN-Festa-dCFR outperformed   On the Zurich dataset, as shown in Table  VII , the Area-WS-AP and Vol-WS-AP, computed without any prior knowledge, as well as the CPWS-APs outperform all other methods in the literature in terms of OA and κ. Our best method, namely Vol-CPWS-AP, outperforms AP by 3. 29%, 2. 26% and 4. 17 in terms of OA, AA and κ, respectively. Considering the classification result per semantic class given in Table  VIII , we observe that the Watershed-APs yielded the best results for most classes, namely roads, buildings, trees, grass and water, but, for the remaining three classes, none of the APs outperformed the baseline NIR+RGB. Concerning the use of prior knowledge, it led to consistent improvements when used in the construction of hierarchical watersheds: all three CPWS-APs outperformed their respective WS-APs by at least 1% in terms of OA, AA and κ.
The most significant improvement was observed for the Dyn-CPWS-AP, which outperformed Dyn-WS-AP by 3. 08%, 2. 21% and 3. 89 in terms of OA, AA and κ × 100, respectively. However, regarding FPWS-APs, we concluded that this method is not well adapted for this dataset and, hence, we do not present land-cover evaluation scores of FPWS-APs on Zurich. The reason is that the Zurich test images do not contain ground truth pixels of all eight semantic classes and, different from CPWS-APs, the FPWS-APs are constructed by considering the classification probability maps of each class individually. For instance, the swimming pool class is present in only two among the five Zurich test images. Hence, the classification probability map for this semantic class would be meaningless for remaining three test images, leading to flat or redundant image reconstructions.
Moreover, APs still falls behind the state-of-the-art FCN model trained on dense annotations (i. e. trained on all pixels of the training set). Fig. 9  illustrates the classification results on the test image zh18 of the Zurich dataset. As already suggested by the scores given in Table  VIII , we can see that none of the APs are able to improve the results for the bare soil class, but most of them provide better classification results for the roads (in black) and grass (in light green). When comparing the WS-APs and CPWS-APs, the most significant improvements are observed on the roads, trees and grass classes, in the regions highlighted by the blue boxes in Fig. 9 (m), (n) and (o). The red boxes indicate a few regions in which classification results were worsened by the use of semantic prior knowledge.
B. Building extraction Taking into account that other methods in the literature, such as  [10] ,  [11]  which also employ prior knowledge for image segmentation, perform well on binary classification/segmentation of images into object and background regions, we aim to validate our Watershed-APs on a binary classification task which is meaningful for remote sensing imagery, namely building extraction. For evaluating our proposed methods on this task, we consider two semantic classes on the Zurich and Vaihingen datasets, namely buildings and background, such that the latter is the union of all remaining classes of each dataset, as mentioned in Section V-B. Evaluating our methods on this binary classification task might give us a clearer idea of their performance, since semantic prior knowledge come from only two classes.