dataesr/scientific-paragraphs-categorization huggingface dataset:
Energy consumption and Peak Hours The data set generated above allows us to determine the energy consumption of each device for each unit of time of one hour. The objective of this section is to summarize this dataset into another one where each column presents the hours of a day and the rows are the different days that are presented in the event file. For example Building 1 in AFNH contains 33 days. The next step is to calculate for each hour in the interval  [1, 24]   energy consumption for each peak is presented in Tables  7  and 8  respectively. For house 5, we notice that the hours 15 a. m, 10 p. m is the peak hours with a frequency of 4 times. Table  7  informs us that the appliance with the highest energy consumption for each peak is the water heater. For house 1, we can see that 18 a.
The MORED datasets contain the energy consumption of different appliances in different neighborhoods. The energy consumption could use it for capturing different Energy Consumption Patterns based on each appliance that is active at each time in the day. MORED is a new datasets that comprises electricity consumption data of various Moroccan premises. MORED provides three main data components: whole premises (wp) electricity consumption, individual load (IL) ground-truth consumption, and fully labeled Il-signature, from affluent and disadvantaged neighborhoods.
Sampling and isolating Saccharomyces cerevisiae From September 2016 to October 2020, we collected a total of 2461 environmental samples from various substrates (bark n = 340, twigs n = 328, leaf n = 528, litter n = 320, and fruit n = 78) surrounding 693 plant hosts (Supplemental Table  S2 ). A total of 339 lichen samples, aliquots from six fermentation practices, and 68 from other sources (insect corpse n = 43, fruiting body n = 14, industrial strains n = 5, and others in which biomaterial was sampled only once n = 6) were also collected. Collection time and GPS coordinates in GPX format of host plants were recorded on the day of collection. Leaves and flowers of host plants were photographed. Bioclimatic variables of sampling sites were retrieved from the CHELSA  (Karger et al. 2017 ) database (v.
1. 2) using recorded GPS coordinates. Digital terrain models (DTMs) of sampling sites were retrieved from Taiwan's Open Government Data website (https://data. gov. tw/dataset/35430). Environmental samples were collected using alcohol-sterilized tweezers or spoons and stored in zip bags. Whenever possible during the sampling trips, metadata such as the identity of the host plant, lichens, and altitude were recorded. Samples were redistributed into 50-mL falcon tubes and stored at room temperature. Each sample was divided into two proportions and immersed in two enrichment media: a liquid medium made up of either (1) 3 g/L yeast extract, 3 g/L malt extract, 5 g/L peptone, 10 g/L sucrose, 7. 6% EtOH, 1 mg/L chloramphenicol, and 0. 1% of 1-M HCl as used previously  (Sniegowski et al. 2002)  or (2) YPD containing 10% dextrose and 5% ethanol adjusted to pH 5.
Sampling efforts were visualized using the R's package ggplot2  (v. 3. 3. 5) and annotated with metR (v. 0. 10. 0; https:// github. com/eliocamp/metR) and ggspatial (v. 1. 1. 5; https:// paleolimbot. github. io/ggspatial/). To determine ploidy levels for our isolates, we performed flow cytometry analysis for the 105 Taiwanese isolates from this study using propidium iodide (PI) staining assay using previously established protocols (Supplemental Information;  Todd et al. 2018).
Amplicons were normalized using the SequalPrep normalization plate kit (Thermo Fisher Scientific A1051001) and then pooled and concentrated using AMPure XP (Beckman Coulter A63881). Finished DNA libraries were sequenced on the Illumina MiSeq platform using 2 × 300-bp pair-end sequencing chemistry. Raw sequencing reads containing the Illumina sequencing index were demultiplexed using sabre (v. 1. 0; https://github. com/najoshi/sabre). Sequencing quality was determined using FastQC (v. 0. 11. 7; https://github. com/s-andrews/FastQC). Reads were quality filtered based on a Qscore > 20, and 50 bp was trimmed from the 3 ′ end using usearch (v. 11. 0. 667) (Edgar 2010). Filtered reads were processed following the UPARSE  (Edgar 2013)  pipeline. In brief, paired reads were merged and dereplicated into unique sequences.
Unique sequences were filtered using usearch default settings. Filtered sequences were denoised into zero-radius operational taxonomic units (zOTUs) using the unoise2 (Edgar 2016b) algorithm. The taxonomy of zOTUs was classified using the SINTAX (Edgar 2016a) algorithm  (Edgar 2016 ) against the UNITE  (Nilsson et al. 2018 ) Fungal database (v. 8. 2). Merged reads were assigned into zOTUs with 100% sequence identity and tabulated using the usearch_global function. Processed reads were analyzed in the RStudio environment (v. 1. 2. 5033). Sequencing data were analyzed with phyloseq (v. 1. 34) (McMurdie and Holmes 2013). Statistical significance was tested for using kruskal. test from the stats package in R (R Core Team 2021).
2018) , 16 isolates from the Brazilian wild lineage  (Barbosa et al. 2016) , and eight isolates from olive brine  (Pontes et al. 2019 ). This combined with the 121 isolates from this study yielded a total of 340 individuals, 30% of which originated from industrial sources and 70% from the natural environment (Supplemental Table  S6 ). Read quality was examined with FastQC (v. 0. 11. 9; https://github. com/s-andrews/FastQC). Read quality and adaptor trimming were performed using Trimmomatic (v0. 36; pair end mode, ILLUMINACLIP; LEADING:20; TRAILING:20; SLIDINGWINDOW:4:20; MINLEN:150)  (Bolger et al. 2014). For the 340 samples, 64%-95% of the raw paired reads from the 340 samples was kept after trimming. Trimmed reads were each mapped to the S288C reference genome version R64-2-1 using the Burrows-Wheeler aligner (v.
0. 1. 5) package  (Fitak 2021) , and presented in Supplemental Information. A consensus genome sequence containing variants for each isolate was generated from the SNPs matrix using BCFtools  (Danecek et al. 2021 ) consensus (v. R64-2-1) with the S288C reference genome sequence. For the IBD analysis, the geographical distance between isolates was measured using the sf package in R for Taiwanese isolates with GPS records. For Chinese isolates, because GPS records were not available, we used approximate coordinates for each sample site  (Duan et al. 2018)  as recommended by the investigators. To estimate the maximum geographical distance within the Chinese lineage, we chose sample sites that were the furthest apart. For instance, for CHN-V, the distance between Shanxi and Hainan was used. For lineages sampled from only one site (CHN-II, CHN-IX), the largest range of the site was used.
Acknowledgments We thank  Cheng-Ruei Lee  for the insightful comments on the ADMIXTURE analyses. We thank  Mao-Ning Tuaumu  for the helpful suggestions on how to deal with bioclimatic variable data. We thank  Nguyen Huu-Vang ,  Cheng-Ruei Lee ,  Jun-Yi Leu ,  Ben-Yang Liao ,  Dang Liu , and  John Wang  for commenting on earlier versions of the manuscript. We thank  Jun-Yi Leu  for experimental advice. We thank  Bo-Fei Chen  and  Ling-Tin Kao  for helping with the initial sampling trips. We thank  Tze-Fu Hsu ,  Yi-Hsiu Kuan ,  H. Thorsten Lumbsch , and  Matthew Nelsen  for collecting/providing some of the biomaterials. We thank  Shou-Fu Duan  and  Feng-Yan Bai  for providing early access to sequencing data and recommendations on how to approximate the geographical positions of the Chinese isolates.
We thank the  National Center for High-Performance Computing  for its computer time and for letting us use its facilities. I. J. T. was supported by the  Ministry of Science and Technology, Taiwan  under grant  110-2628-B-001-027  and  Career Development Award   AS-CDA-107-L01 ,  Academia Sinica. Authors contributions: I. J. T. conceived and led the study. T. J. L. , Y. -C. L. , and W. -A. L. performed the sampling and isolation of Saccharomyces cerevisiae. J. -P. H. , C. -L. H. , and K. -F. C. helped with the sampling and identified the lichen and plant samples. T. J. L. , W. -A. L. , Y. -F. L. , and H. -M. K. conducted the experiments. Y. -F. L. performed the amplicon analyses. H. -H. L. , H. -M. K. , and I. J. T. performed the sequencing and assemblies of the S. cerevisiae genomes. T. J.
Study Participants The Kangbuk Samsung Health Study is a cohort study of Korean men and women aged ≥18 years who participated in comprehensive health examinations every 1-2 years at Kangbuk Samsung Hospital Total Healthcare Center in Seoul and Suwon, South Korea, as previously described  11. The present cohort study included participants who underwent comprehensive health examinations between January 2012 and December 2018. From 2012, data on appendicular skeletal muscle mass and serum 25(OH)D levels were available, and all participants had at least one follow-up visit between recruitment and December 31, 2020 (n=208,026 participants). A total of 39,656 participants were excluded in a two-step selection process (See Supplementary Material for detailed exclusion criteria) This study was approved by the Institutional Review Board of Kangbuk Samsung Hospital (IRB No.
Statistical Analyses The primary outcome was the development of incident LMM. Each participant was followed from the baseline visit until either the occurrence of incident LMM or the last health examination conducted through the end of 2020, whichever occurred first. The incidence rates were calculated as the number of incident cases divided by person-years of follow-up. Cox proportional hazard models were used to estimate the hazard ratios (HRs) with 95% confidence intervals (CIs) for incident LMM in each 25(OH)D category compared with the reference category, while using three models with progressive adjustment to control for potential confounders (See Supplementary Material). STATA version 16. 0 (Stata Corp. , College Station, TX, USA) was used for data analysis. All P-values were two-tailed, and P-values <0.
Itemset mining methods are techniques to discover relevant patterns in transactional databases. The first approach, called constrained-based pattern mining, is based on exhaustive pattern mining techniques which consist in returning all itemsets that satisfy a given constraint. The main issues that hinder their efficiency are the pattern explosion and the difficulty for a user to set the threshold value. To solve this problem, methods that return the most interesting patterns, called top-k, are also proposed, but they tend to lack diversity, a challenging issue for interactive pattern mining. However, interactive pattern mining is based on fast methods that respond effectively to user demand. To overcome all these problems, output pattern sampling is proposed to draw quickly a set of interesting patterns while guaranteeing a good diversity.
Pattern sampling techniques are probabilistic methods that aim to draw a set of interesting patterns where each pattern is drawn with a probability proportional to a given interestingness measure. Nowadays, there are several measures that a user can test when interacting with the same database. In that case, the system should last a few times to consider the new utility measures while guaranteeing an exact draw. So, the cost in time of utility change can be a real problem for output pattern sampling techniques in large databases. In addition, with the current sampling methods, it is necessary to store all data in memory and this storage is prohibitive for large data. To solve these problems, this paper deals with how to structure the data for output pattern sampling under length-based utility measures in large transactional databases.
So, we revisit the trie structure initially proposed by D. Knuth to enrich it and then no longer have the need (i) to access the data to sample because the patterns will be directly taken from the enriched trie, (ii) to reprocess the entire dataset when utility changes. The computation of the value of a length-based utility measure is based on the lengths of all the patterns that are present in the database. So, we define a new structure of trie called trie of occurrences, built by our first algorithm TPSpace (Trie-based Pattern Space), which materializes all the occurrences of the patterns in the database.
Diop et al The data compression comes from a factorization of the information via the prefixes of the patterns. The particularly remarkable result is that, by definition, the trie of occurrences is the same for any length-based utility measure provided that the same values are kept for the minimum and maximum length constraints. We then describe TPSampling (Trie-based Pattern Sampling) which performs the sampling by drawing patterns according to a length-based utility measure from the trie of occurrences. This paper is completed by the complexity analysis in memory and in time of the method and experiments on benchmark datasets. TPSampling is competitive with the twostep approach to sample following a given interestingness measure but, as expected, it is more particularly advantageous if several utility measures are used thanks to the generic preprocessing.
Introduction Pattern mining  [1]  is an active research area that aims at discovering interesting and not trivial information in large databases. Itemset mining methods are techniques to discover relevant patterns in a transactional database. During the last decade, the researchers in this field addressed the pattern explosion challenge resulting from the joint effect of the volume of data and the combinatorial nature of the mining methods. In fact, it is very difficult to control the size of the set of frequent patterns given a minimum threshold for instance. On the one hand, if the minimum threshold is very small, the set of returned patterns overwhelmed the end-user. On the other hand, if the minimum threshold is very large, the set of patterns risks being empty. To solve this problem, many approaches are proposed like Top-k pattern mining  [2]  which returns the k most frequent patterns, but with a lake of diversity.
The recent proposal approach is based on output pattern sampling  [3]. Output pattern sampling consists in generating a sample of patterns among the patterns that would have been extracted from the complete dataset. It is a non-exhaustive method to extract relevant patterns and ensures good interaction and great diversity while offering strong statistical guarantees thanks to its random nature. Its usefulness has been widely demonstrated in recent years in many areas like feature classification  [4] , outlier detection  [5]  or interactive discovery  [6, 7, 8]. It has also been applied to several pattern languages such as graphs  [3] , itemsets  [9, 4, 10]  and sequences  [11, 12]. However, at the time of writing this paper, none of the current sampling methods does mind managing the size in memory of the data or the reprocessing when the user changes the given utility.
In the field of pattern mining, it is also very important to consider the size of the database which can be an obstacle when the data must be loaded into memory. Indeed, with the output sampling methods of the state-of-the-art, it is a necessity that the entire database is stored in memory except using a decentralized approach  [10]  where the data are natively decentralized in different sites. In our case, we assume that all transactions are available in a single and unique machine where the user needs to run a sampling algorithm. In some datasets with very large sizes, one can use a compact data structure in order to have a compressed representation of the database as it was already done by the exhaustive extraction of interesting patterns  [13]. To the best of our knowledge, none of the output pattern sampling methods in the literature  [3, 4, 9, 11 ] has yet been applied to compact representations of databases.
However, most of the proposed methods need to store the entire database in memory. It is also very important to facilitate reprocessing when a user decides to change a utility measure. So, an efficient method should not weight each transaction at each change. The processing phase is often consuming when the size of the database becomes large. In  [14]  the authors tackle the problem of utility change when they work with many utilities like frequency, area, or decay. But, the proposed solution also depends on the number of transactions in the database. Then, it has the same complexity of reprocessing as the Two-Step pattern sampling in  [4]. So, besides the need of reducing the storage cost, this paper addresses techniques to speed up the reprocessing times for pattern sampling in large databases when the user changes the interestingness measure.
The genericity of TPSampling comes from the fact that it can consider any interestingness measure based on the length. -We theoretically and experimentally evaluate the complexity of TPSampling. In particular, we show that TPSampling makes an exact draw according to a length-based utility measure chosen by the user and directly from the trie. In addition, we evaluate the complexity in memory storage of the trie of occurrences on different transactional databases including synthetic datasets and then we evaluate its speed according to different length-based utility measures. The remainder of this paper is organized as follows: Section 2 situates our work in the state-of-the-art of pattern sampling methods and highlights the main motivations behind this paper. Section 3 begins with some basics to fully understand our approach to drawing patterns from a trie.
Random walk sampling. The first class of methods uses random walks in the search space to sample interesting patterns. In this class, most of the methods  [3, 16, 15, 18]  are based on Markov Chain Monte Carlo (MCMC) methods. The idea is to build a Markov chain simulating a distribution of a probability law. In  [3] , the authors proposed the first method of this class to sample frequent sub-graphs in a graph database, and then  [16]  proposed another method for interactive discovery. One of the limits of these methods is the slowness of their convergence. Efficiency in memory storage: It is important to note that all these algorithms run locally in a single machine. So, they fully benefit from the available memory by storing the entire data. Flexibility on utility change: The methods of this class need to restart from scratch when the user changes the utility measure.
Indeed, the partial order graph of the interesting sub-graphs must be readjusted to make new random walks. SAT-based sampling. The second class of output sampling methods is based on the SAT logical formalism. It has been implemented for the output sampling of pattern set  [9]. In this article, the authors have presented both a generic algorithm called GFlexic and a specialized algorithm called EFlexic. For this class of methods, the basic idea is to use constraint programming and logical solvers to perform pattern mining. On a practical level, GFlexic has only been tested on small transactional databases (at most 4,000 transactions and 300 items), which is explained by the very great genericity of the method. One of the limits of these methods is that they are not exact (the draw of a pattern is approximatively exact).
Efficiency in memory storage: Like the random walk methods, all these algorithms run locally in a single machine and need to use the available memory for storing the entire data. Flexibility on utility change: These methods need also to restart from scratch when the user changes the utility measure. Indeed, they have to divide the space of the valuations to satisfy the newly obtained logical formula. Multi-step sampling. The multi-step approaches  [4, 17, 22, 11, 12, 10, 14]  walk in two phases. The first phase is dedicated to the preprocessing which consists to weight each instance (transaction or sequence) of the database proportionally to the sum of the interest of the distinct patterns it contains. The multi-step sampling methods presented in  [4, 17, 22, 11, 12]  deal with local database in a single machine.
However, in  [10, 14] , a multi-step algorithm is designed for transactional distributed databases and was applied in DBpedia and Wikidata. This algorithm centralizes the length of the transactions and uses two primitives to construct the sampled pattern. It is then important to note that even if we readapted the proposed algorithm in  [10, 14]  for local data, it could in no way reduce the used size in memory. Efficiency in memory storage: These methods also need to store the entire data in main memory except that in  [10, 14]  in the case of native distributed databases. Flexibility on utility change: When the user changes the utility measure, the twostep methods do a reprocessing phase which consists of weighting each transaction of the database according to the new utility measure. In  [14] , the authors show that the reprocessing phase is experimentally fast because proportional to the number of transactions in the database.
Data structures for pattern mining To solve the problem of pattern mining, many data structures have been proposed such as "FP-Tree"  [13]  or "Trie"  [27]. In transactional databases, there are lots of repetitions because several transactions can contain the same information. These repetitions make sense in the field of pattern mining because they allow us to find interesting rules, but we have to know how to represent them. For example, in a transactional database, if 90% of transactions that contain the items {e 1 , e 2 , e 3 , e 4 } also contain the items {e 5 , e 6 } then we might as well group them together so that they share the same prefix. This considerably reduces the size of the database in memory. With "FP-Tree" or "Trie", it becomes possible to represent transactions containing the same item in one path.
This is because they share the same prefix. But the difference between these two structures is that, unlike "trie" which only links a node with its children, "FP-Tree" establishes links between nodes of different branches to quickly compute the frequency of the patterns. Since we do not want to compute this latter, we suggest using "trie"  [27]  to have a compact representation of the database in memory. The "Trie" data structure is widely used in the field of text mining  [28]. Other authors have used the trie structure for frequent pattern mining with Apriori  [29]. Through the previous works, we note that trie is much more parsimonious in storage cost on transactional databases than on text databases. This is because the depth of the tree depends exclusively on the size of the longest chain.
Basic definitions Let I = {e 1 , • • • , e N } be a finite set of literals called items. We assume that there is an arbitrary total order > I between items : e 1 > I • • • > I e N. An itemset (or pattern), denoted by ϕ = {e i1 , • • • , e in } (or simply ϕ = e i1 • • • e in ), with n ≤ N , is a none empty subset of I, ϕ ⊆ I. The set of all patterns that we can generate from I is called the pattern language denoted by L = 2 |I| \ ∅. The length of a pattern ϕ ∈ L denoted by |ϕ| is the number of items it contains (its cardinality). A transactional database T is a multi-set of itemsets (called transactions) where each of them has a unique identifier j ∈ N. We denote by t j = e 1 • • • e n a transaction identified by j of length |t| = n defined in I, and L(T ) the set of all patterns that appear in T.