dataesr/scientific-paragraphs-categorization huggingface dataset:
The MOBY data set provides a 23 year time series that can be used to validate the shadow correction. To test the shadow correction factor, S ′ (θ o , φ, W), we binned the data set by θ o and φ. We focused on the normalized water leaving radiance, Lwn, to reduce variability due to illumination conditions. Ignoring polarization sensitivity, which is a small factor in MOBY, S ′ (θ o , φ, W) should be the major cause of an asymmetry of Lwn with φ. We will also concentrate on Lwn1, which is the product with the smallest uncertainty, as it requires a smaller factor to propagate Lu(1 m) to the surface as opposed to Lu(5 m), and also derives the K L from the two Lu values closest to the surface. We will focus on 2 wavelength bands, integrated over channels of the Visible and Infrared Imager/Radiometer Suite (VIIRS) satellite sensor  [26] , 444.
For φ greater than 160°, Lwn1 decreased at all solar zenith angles, but particularly strongly at larger θ o. For φ from 0°to 160°there is a little variation, but nothing consistent. In contrast, Lwn1 at 556. 3 nm is fairly stable, until φ = 160°. Beyond 160°, however, data at 556. 3 nm decreased with increasing θ o. The error bars reflect the standard deviation of the data which went into the average values of each bin. The structure in Lwn1 at 444. 5 nm, with maximum values at θ o ∼30°needs to be explained. The explanation can come from how MOBY data were collected. In general, MOBY collects data three times a day, at fixed times not fixed θ o. Given fixed data collection times, θ o for a given measurement time will vary seasonally, so the variation with θ o in the data set also reflects a seasonal variation in water properties at the MOBY site.
On the x-axis, the label represents the center of the 10°bin. Days for which there were two or more good (cloud free) measurements on the same day provide a check on this interpretation. For each of the days in the MOBY dataset for which there were two or more good measurements, we normalized the BRDF corrected measurement of Lwn1 at the higher θ o by the BRDF corrected measurement of Lwn1 for the measurement at the smallest θ o for that day. Assuming the water properties did not change significantly and consistently during a single day between these two measurements, one would expect that, on average, there should be no difference in the corrected Lwn1 during that day. Figure  7  shows the result of this comparison. A linear best fit of the unitless ratio of normalized Lwn1 pairs to the relative solar zenith angle of the measurements in each pair had a slope not significantly different from 1.
The 2-D correction was based on at least two factors that might account for this. First, the 3-D correction is done at higher spatial and angular resolution (3°azimuthal resolution vs 15°for the 2-D model). Second, when investigating which part of the structure is causing the shadow, it is obvious, from the geometry, that in the 3-D model, the surface buoy's vertical structure causes much more shadowing that is not captured in the 2-D model. This correction will be made available for the MOBY water leaving radiance data set, along with the uncertainty in the correction. As discussed, for azimuths less than 150°, the correction is small. In general, for azimuthal angles greater than 150°, the uncertainty of the correction was less than 20% of the correction. We are working towards determining a measurement by measurement wavelength dependent uncertainty for the entire MOBY data set, which will include this shadowing factor.
Introduction With the rapid development of 5G, the Internet of Things (IoT), and artificial intelligence (AI) in recent years, increasing numbers of large datasets are becoming available in a wide variety of communities. In this respect, identifying the cohesive structures in these various datasets facilitates the discovery of valuable hidden patterns. Formal concept analysis (FCA) provides a robust mathematical foundation based on lattice theory for identifying the cohesive structures of a social network  [1, 2]. In social network analysis, we can model the topological information as a bipartite graph; i. e. , a graph with two types of vertices and whose links are only between vertices of different types. Then, by identifying the formal concepts (also known as biclusters  [3]  or bicliques  [4] ) from bipartite graphs, the network structure is transformed into a concept form, which indicates the relationships hidden in the data.
The algorithm was mainly guided by the quality of the extracted association that may be drawn from the intent part of the selected formal concepts. More specifically, the aim was to reduce the number of mined patterns to make them manageable for the end users while preserving the quality of the mined knowledge. The main criticism of this algorithm is the running time, especially when handling large datasets. In addition, the algorithm provides only full conceptual coverage, which is not always needed. In this paper, along the same lines, we introduce the greedy and parametrizable algorithm called CONCISE. We show that CONCISE is competitive in terms of running time and complexity. The main broad points of the fresh approach that we introduce are as follows: 1. Extraction of full and partial conceptual coverage: Our approach finds a minimal subset of formal concepts that fully or partially cover the relations in the formal context.
Essential Formal Concepts Essential concepts, also called mandatory concepts (MCs), play a crucial role in data mining as they allow the discovery of regular structures from data based on formal concept analysis (FCA). They qualify as essential because they belong to any conceptual coverage of a formal context  [22]. From the relational algebra (RA) perspective, an essential concept contains at least one isolated point, as introduced by Riguet  [30]. As a mathematical background, FCA and RA have already been combined and used to discover regularities in data  [18]. A formal concept represents the regular atomic structure for decomposing a binary relation. Moreover, the computing of Riguet's difunctional relation  [30]  results in a set of isolated points describing invariant structures that could be used for database decomposition and textual feature selection (TFS)  [19].
Hence, from a "worst case" context of a dimension equal to n×(n+1), 2 n formal concepts can be extracted. Even if the worst case is rarely encountered in practice, "worst case" datasets have been shown to allow the behavior of an algorithm to be scrutinized on extremely sparse concepts and hence to assess its scalability  [15]. Table  4  presents an example of a "worst case" dataset for n = 4. a b c d 1 × × × 2 × × × 3 × × × 4 × × × 5 × × × × Corollary 3. No essential concepts can be extracted from a worst case formal context. Proof. Let us consider a worst case formal context K = (O, I, R). By constructing a worst case dataset, and with regard to Theorem 2, we have the following assumptions: • ∀o ∈ O, |{o} | = 2 • ∀i ∈ I, |i| = n ∀(o, i) , we find always that |{o} | = |{i} |. Thus, no essential concepts can be drawn from a worst case dataset.
Algorithm 1: The CONCISE algorithm. The CONCISE algorithm proceeds according to the following steps: Input: Formal context K = (O, I, R), δ: coverage threshold Output: Conceptual coverage C K begin C K ← ∅; /* step 1: Detect the essential concepts */ COMPUTE_INTRODUCTORY_CLOSURE (K); C K ← COMPUTE_ESSENTIAL_CONCEPTS (K); if ( IS_COVERED (K)< δ ) then /* Step 1: Detect the essential concepts After closing the items through the COMPUTE_INTRODUCTORY_CLOSURE procedure, the efficient detection of the set of essential formal concepts (if they exist) is conducted by the COMPUTE_ESSENTIAL_CONCEPTS function. The corresponding pseudocode is provided by Algorithm 2. The algorithm iterates over the seed set of attributes I. In (Lines 4-8) and with regard to Corollary 2, if the cardinality of the extent part is equal to 1, then its induced formal concept is considered an essential concept, and we remove all the covered elements from the formal context.
Benchmark Datasets In this study, we used some benchmark datasets for experimental investigations of the performance and robustness of our proposed algorithm. As shown in Table  7 , we considered the Apj and Americas-small datasets. The remaining datasets were furnished by the UC Irvine Machine Learning Database Repository  [34]. The table presents the number of objects, the number of attributes, and the number of all formal concepts that may be drawn from the dataset using the LCM algorithm  [35]  for each dataset. The datasets are listed in increasing order with regard to the number of formal concepts.
• The Neumann neighborhood (N Stress) comprises the (at most) four adjacent entries resulting in the local stress of x ij : N ij = min(n,i+1) ∑ k=max(1,i-1) (x ij -x kj ) 2 + min(m,j+1) ∑ l=max(1,j-1) (x ij -x il ) 2. ( 11 ) As depicted by Table  8 , the CONCISE algorithm gives equal or more compact coverages than the GREESS algorithm on 12 out of 16 datasets. Furthermore, for the Soybean-large and Dermatology datasets, CONCISE outputs 103 and 128 formal concepts, respectively, while GREESS flags 126 and 158 formal concepts, respectively. A close look at Table 8 reveals that CONCISE performs better than GREESS (except with the Mushroom dataset) when N stress and M stress are higher. Although the GREESS algorithm outperforms CONCISE for some datasets, the latter could not provide results for Americas-large, Dual-matching-40, and Ac-90k datasets.
Comparison between the Full and Partial Coverage of the CONCISE Algorithm The principal added value of the CONCISE algorithm is that it provides full and partial coverage of formal concepts using a threshold δ. We evaluate the obtained coverage regarding the number of concepts, quality metrics, and running time below. The impact of the variation of δ on the number of concepts obtained by the CONCISE algorithm is shown in Figure  2  and Table  9. Figure  2  shows that the number of concepts decreases drastically when switching from full coverage (δ = 1) to partial coverage (δ = 0. 9). For example, the Apj dataset is completely covered by 774 concepts, while only 321 concepts are needed to cover 90%. This difference is smaller between the different thresholds of the partial coverages. The Americas-large dataset is covered by 182 concepts when δ = 0.
8, and only 10 concepts are omitted when δ = 0. 7. The number of concepts remains the same for the Americas-small dataset from the threshold δ = 0. 7. The Ac-90k dataset represents a particular case because the first found concept covers approximately 90% of the formal context. In the following, we evaluate the CONCISE algorithm in terms of quality. Several measures for concept interestingness were recently reviewed by Kuznetsov et al. [37]. In this study, we use the two most common measures, which are stability and separation  [28]. Then, we propose a new measure called object uniformity. Definition 13 (STABILITY). Stability seems to be the most widely used metric in the FCA community and is applied in numerous applications  [38] ; e. g. , biclustering and the detection of scientific subcommunities, among others.
In our experiments, we used the DFSP algorithm  [40]  to compute the stability of the obtained coverage. This method is considered an efficient algorithm for computing the exact stability. Table  10  shows that the CONCISE algorithm obtains excellent stability values, especially on the Apj, Breast-cancer, and Tic-tac-toe datasets, where the stability is higher than 0. 8. Moreover, we should also mention that for most of the datasets, the stability of the coverage is better for partial coverage. For example, the stability for the Americas-small dataset ranges from 0. 598 for δ = 1 to 0. 779 for δ = 0. 5. However, CONCISE obtains bad results on the Chess, Dual-matching-40, and Ac-90k datasets, even with lower thresholds, and does not exceed the rate of 0. 189. Definition 14 (SEPARATION). The separation metric  [41]  is meant to describe how well a concept sorts out the objects it covers from other objects and how well it sorts out the attributes it covers from other attributes of the context.
Thus, this metric characterizes how specific the relationship between the objects and attributes of the concept is concerning the formal context. For example, the separation index of the formal concept A, B is defined as follows: s( A, B ) = | A | × | B | ∑ a∈A | {a} | + ∑ b∈B | {b} | -| A | × | B |. ( 14 ) The higher the separation index of a concept, the smaller the number of similar concepts in the formal context. It is defined as the ratio between the area covered by the concept and the total area covered by its objects and attributes. The results of the separation metric are described in Table  11 , which shows that the best separation rate is not always obtained with the same threshold for the different datasets. For example, considering the Apj Americas-small Paleo and Americas-large datasets, the separation is better when δ = 0.
9. However, for the DBLP, DNA, Mushroom, Soybean-large, and Chess datasets, the maximum separation is obtained with thresholds equal to 0. 6 and 0. 5, respectively. Note that varying the threshold does not affect the separation value for the House-vote, Tic-tac-toe, and Dual-matching-40 datasets. In the following, we introduce a new quality metric of formal concepts called object uniformity. Definition 15 (OBJECT UNIFORMITY). We know that the intent part is the maximal set of attributes located at the intersection of all the objects of the extent part. If we consider each object of the extent part, we would like to assess to what extent the pseudoconcept is different from the formal concept X, Y. Please note that all of these pseudoconcepts share the same extent part. To assess such uniformity or cohesion, we introduce the following metric called object uniformity.
If we consider the formal concept C = X, Y , then we define the following metric: Q1(C) = ∑ o∈X |X|×|Y| |X|×|o | |X| (15) Example 12. Let us consider the formal concept C 1 = 13456, a extracted from the formal context given by Table  2. Then, as shown by Table  12 , we have: Q1(C 1 ) = 5×1 5×2 + 5×1 5×2 + 5×1 5×4 + 5×1 5×2 + 5×1 5×2 5 = 0. 45 If we also consider the formal concept C 2 = 134, ad , then we have according to Table  13 : (X, 1 ) (X, 3 ) (X, 4 ) Q1(C 2 ) = 3×2 3×2 + 3×2 3×2 + 3×2 3×4 3 = 0. 83 134 × ad 134 × ad 134 × abcd Table  14  shows the obtained results of the object uniformity with the different thresholds. Similar to the separation metric, there is no fixed threshold that gives the best results for all the datasets. For example, better results are obtained on the Apj, Americas-large, Soybean, and Chess datasets with a threshold equal to 0.
5. Conversely, better results are obtained on the Breast-cancer, Paleo, Spect-test, Mushroom, and Dermatology datasets with full coverage. It is also important to mention that, on average, there is no significant difference between the obtained results when varying the thresholds. For instance, this difference is equal to 0. 001 on the House-vote and Ac-90k datasets. Table  15  shows that the proposed algorithm is very efficient and provides excellent results with all thresholds. For example, the proposed algorithm can process 636 and 474 concepts in 0. 52 and 0. 809 s, respectively, on the Americas-large and Apj datasets. Furthermore, the proposed algorithm has the highest running time on the Dual-matching-40 dataset among all datasets, and the dataset was handled in 1011. 790 s in the worst case.
Moreover, it is essential to point out that the GREESS algorithm was unable to handle the same dataset within 48 h. We did not compare the running times of the two algorithms because they were not implemented using the same programming language. The efficiency of the CONCISE algorithm is due to us using the C++ language and parallelism paradigm in implementation. The source code is publicly available at https://github. com/AmiraMouakher/Concise (accessed on 15 September 2021).
Conclusions and Perspectives This paper proposed a greedy approximation algorithm, called CONCISE, to find a minimal subset of formal concepts that fully or partially cover the formal context's relations. The proposed method avoids computing the entire set of formal concepts associated with a given formal context. Moreover, the CONCISE algorithm yielded high quality for both full and partial coverage in a reasonable running time, even for large datasets. In the near future, we plan to pay close attention to the following issues: 1. Shallow embedding: From "Boolean matrix factorization," the presented concise coverage leads to the establishment of a gainful approach for unveiling the smallest set of hidden factors, also known as shallow embedding, in contrast to the deep approach learned by deep learning-based techniques.
Although RNA sequencing has been becoming the main transcriptomic approach in the model legume Medicago truncatula, there is currently no genome-wide gene expression atlas covering the whole set of RNAseq data published for this species. Nowadays, such tool is highly valuable to provide a global view of gene expression in a wide range of conditions and tissues/organs. Here, we present MtExpress, a gene expression atlas that compiles an exhaustive set of published M. truncatula RNAseq data (https://medicago. toulouse. inrae. fr/MtExpress). MtExpress makes use of recent releases of M. truncatula genome sequence and annotation, as well as up-to-date tools to perform mapping, quality control, statistical analysis and normalization of RNAseq data. MtExpress combines semi-automated pipelines with manual re-labelling and organization of samples, to produce an attractive and user-friendly interface, fully integrated with other available Medicago genomic resources.
Importantly, MtExpress is highly flexible, in terms of both queries, e. g. allowing searches with gene names and orthologous gene IDs from Arabidopsis and other legume species, and outputs, to customize visualization and redirect gene study to relevant Medicago webservers. Thanks to its semi-automated pipeline, MtExpress will be frequently updated to follow the rapid pace of M. truncatula RNAseq data publications, as well as the constant improvement of genome annotation.
2016). This notably included a gene expression atlas (MtGEA; https://mtgea. noble. org/v3) based upon microarray technology developed by Affymetrix and representing 33,229 genes from the genome version 3  (Kang et al. 2016). This atlas integrated 270 different samples (not considering biological replicates) in its last release and has been extensively used by the Medicago community. Unfortunately, MtGEA website was not maintained and no longer responds to inquiries. In the past years, RNA sequencing has become increasingly popular to study gene expression at the plant, organ and tissue levels. The advantages of this approach are its sensitivity and its flexibility to exploit new genome releases and annotations by the possibility of (re)mapping sequencing reads, as well as to discover transcript isoforms or novel transcripts such as long noncoding RNAs.
A flexible and user-friendly interface Thanks to synonymy links previously computed in the LeGOO knowledge base  (Carrère et al. 2019) , MtExpress can be queried by any gene locus identifier from the different M. truncatula genome releases (from Mt3. 0 to Mt5. 0), and identifiers of several microarray generations, namely Oli16k  (Kuster et al. 2004 ), Affymetrix  (Benedito et al. 2008)  and Nimblegen  (Verdier et al. 2013). Moreover   (Kreplak et al. 2019) , can be used to get the expression pattern of putative orthologues in M. truncatula  (Carrère et al. 2019). The query output (Fig. 2 ) was set up to display by default the expression profile of the queried gene using a 'reference dataset', containing 28 samples which provide an overview of gene expression in various plant organs as well as symbiotic interactions.
Then, the user can switch to one of the 18 pre-arranged categories, each with a more extensive or specialized set of samples: e. g. 'abiotic factors' and 'biotic stress' categories with all RNAseq samples related to different stresses, or 'seed' and 'nodule' categories, dedicated to samples specific to these organs. If needed, the complete dataset containing the 431 samples can also be selected for visualization. Finally, custommade selections can also be defined by the user by selecting/unselecting specific experiments to visualize and export the desired graphical outputs. All of the information associated to each experiment is provided, including the MultiQC  (Ewels et al. 2016 ) report, and the corresponding publication, metadata and raw counts. Gene expression levels (with a choice between log2 TMM or raw TMM values) are displayed as box plots, with various informative elements (notably project name from SRA and descriptive statistics from biological replicates such as mean, median, min, max, Q1, Q3), each project being distinguished by a different color.
Additional information on queried genes is easily accessible via the 'synonymous' and 'annotation' buttons, as well as links to the Mt5. 0 genome browser and the LeGOO knowledge database. Conversely, a link to MtExpress is provided for all annotated genes in the Mt5. 0 genome browser. To validate the overall pipeline, we examined the expression profile of a number of gene markers described to be specifically expressed in certain organs, tissues, or conditions. We found that their expression pattern was as expected from the literature. Three of them are shown in Fig. 3 , using the reference sample set: MtPT4 (PHOSPHATE TRANSPORTER 4)  (Harrison et al. 2002).
Material and Methods Collecting Data RNAseq dataset. We collected all data available as of April 1st 2021 in the Sequence Read Archive (SRA), using the Run Selector with the query "Medicago truncatula [ORG]" and the filter "AssayType = RNA-seq". We kept all studies associated with a peer-reviewed publication, representing 51 projects altogether (supplemental Data set 1). and listed in csv files with curated sample names, replicates and library strandedness to be properly processed. Affymetrix microarray MtGEA dataset. We exported experiment replicates for all probesets (including Mtr, Msa, Sme and control probesets) from https://mtgea. noble. org/v3/experiments. php downloading webpage  (Benedito et al. 2008). We excluded a few duplicated libraries and added one missing set of published data (supplemental Data set 1).
All samples were manually ordered, classified into 18 categories (supplemental Data set 1) and aliases were added to homogenize labelling. Metadata such as publication, organ, experimental factors were manually extracted from the MtGEA website. Expression Measure of RNAseq dataset nf-core/rnaseq version 3. 0  (Patel et al. 2020)  was used with default parameters to run expression measure separately on each SRA project (Fig. 1 ). We chose STAR  (Dobin et al. 2013)  as read mapper and SALMON as read quantifier  (Patro et al. 2017). We used the version 5. 1. 7 of M. truncatula A17 genome annotation  (Pecrix et al. 2018)   Sinorhizobium medicae WSM419, Fusarium oxysporum f. sp. medicaginis, Rhizophagus irregularis, Ralstonia pseudosolanacearum GMI1000, Verticillium albo-atrum V31-2, Gigaspora rosea and Rhizoctonia solani AG-3) (supplemental Data set 1).
Normalization RNAseq dataset. The complete dataset (51 SRA projects representing 431 samples) was normalized together in order to provide consistent values across gene expression comparisons in sub-categories, using TMM normalization from EdgeR package  (Robinson and Oshlack 2010). Affymetrix microarray MtGEA dataset. The expression data downloaded from the MtGEA site website were already normalized using a robust multichip average method (RMA normalization).