dataesr/scientific-paragraphs-categorization huggingface dataset:
Mining Extracting itemsets using jLCM Generating association rules, presented in Section 2. 2, requires to first extract frequent itemsets from T. We use j LCM  [16] , our open-source parallel and distributed pattern mining algorithm that runs on MapReduce  [13]. Mining frequent itemsets is done in two steps. We scan the input dataset T once and build a filtered dataset limited to transactions containing the target B specified by the analyst: T B = {E ∈ T , B ∈ E}. Then, we execute j LCM on the filtered dataset. j LCM is a recursive algorithm that retrieves frequent itemsets and computes their frequency. Closed itemsets are returned along with their corresponding support, except for singletons that cannot be used to produce association rules. This extraction allows us to quickly obtain itemsets that satisfy our constraint.
, i. e, all extracted itemsets contain the specified target B. Mining rules Our analysts aim at uncovering interesting association rules expressed as A → B where B is the specified target. Evaluating the interestingness of an association rule requires computing the support of itemsets A, B and A∪B in T. The standard method for mining association rules consists in finding all frequent itemsets in the dataset, and then generating the rules. Given that our analyst specifies a single target B at a time, this approach would be wasteful. This motivates using j LCM on the filtered dataset limited to transactions containing the target B. The result of the itemsets extraction using j LCM contains the support of B and A ∪ B for all association rules we are interested in. At this point, we need to calculate the support of each antecedent itemset A.
Thus, in a post-processing step, we scan the dataset T once and compute the support of all antecedents A. This two-step approach avoids the computation of many itemsets that will never appear as a rule antecedent. Evaluating relevant rules To evaluate the interestingness of an association rule A → B, we only need to compute P (A), P (B) and P (A ∪ B) because given the number of all transactions |T |, other probabilities such as P (B|A) and P (A¬B) can be derived. Therefore, we denormalize the results of the mining phase to store those three probabilities with each A and B. The support of all rules' antecedents (used to compute P (A)) are added to the results of the mining phase (used to compute P (B) and P (A ∪ B)). We create a dataframe where each row represents an association rule and has enough information to compute its interestingness.
Ranking and summarization Our goal, stated in Section 2. 3, is to assist analysts in selecting the most actionable rules, those that can be used to promote products or target specific customers. In this section, we present an empirical evaluation of the 35 measures for association rules introduced in Section 2. 2. The main goal of our evaluation is to compare the rankings of association rules produced by those measures on our dataset, and study their similarities. This lets us summarize ranking measures into similar clusters. We explain obtained clusters in Sections 4. 2 and 4. 3 and discuss their differences. This empirical evaluation automatically reduces the number of candidate measures to present to analysts in the user study.
For our hierarchical clustering implementation, we rely   3 ) on the cluster. hierarchy function available from the scipy statistics package of Python. We obtain a dendrogram of interestingness measures and analyze their similarities. The dendrograms for N DCC and τ are presented in Figure  4. Figure  5  shows the complete dendrogram for all interestingness measures using hierarchical clustering. To describe the results more easily, we partition the interestingness measures into 5 clusters, as indicated in the third column in Table  3. G 1 is by far the largest cluster and contains 18 measures (among which Lift, Confidence, Added value) that produce very similar rankings, among them 6 clusters of measures always generate the same rankings. A second cluster G 2 comprising 3 measures (Accuracy, Gini index, Least contradiction) is similar to G 1 according to τ.
Running time and memory consumption Our development environment is comprised of Python 3. 7. 0 that invokes j LCM (implemented in JDK 7), for each target product or category on a 2. 7 GHz Intel Core i7 machine with a 16 GB main memory, running OS X 10. 13. 6. Table  6 ) presents the average running time as well as the memory consumption of prod assoc over 228 target products and that of demo assoc over 16 categories. We note that demo assoc runs slower than prod assoc. This is mainly due to the difference in cardinalities of the constructed transactional datasets: 35, 377, 345 transactions in demo assoc and 1, 083, 901 transactions in prod assoc. We notice a similar trend regarding memory consumption.
Actionable is interpreted as the most likely to lead to relevant recommendations. This experiment involved 2 experienced analysts: one data scientist and one product manager (co-authors of this paper). For each mining scenario, prod assoc and demo assoc, we sampled target products and target categories respectively. Each analyst picks a mining scenario among prod assoc and demo assoc for which a target product or a target category must be chosen, respectively. The analyst receives a ranked list of rules. Neither the name of the measure nor its computed values for association rules are revealed because we wanted analysts to evaluate rankings without knowing how they were produced. For a given scenario and a target product or category, our analysts completed 20 comparative evaluations showing two rankings to be compared with the top-10 rules per ranking.
Recommendation through Association Rules Recommendation systems can benefit from association rules extraction  [11, 26]. As shown in the experimental study by Pradel et al. [24] , association rules have demonstrated good performance in recommendations using real-world ecommerce datasets, where explicit feedback such as ratings on the products is not available. Thus, it appears necessary to evaluate the performance of recommendations based on association rules mining on our dataset. Association rules were first used to develop top-N recommendations by Sarwar et al  [26]. They use support and confidence to measure the strength of a rule. First, for each customer, they build a single transaction containing all products that were ever purchased by that customer. Then, they use association rule mining to retrieve all the rules satisfying a given minimum support and minimum confidence constraints.
Experiments Protocol In this section, we present our experimental protocol and the evaluation measures we use in our experiments. The widely used strategy for evaluating recommendation accuracy in offline settings is to split the dataset into training and test sets. The test set is used to simulate future transactions (ratings, clicks, purchases, etc) and it usually contains a fraction of transactions. The remaining interactions are kept in the training set and are fed to the recommendation algorithm to output a list of top-N product recommendations for each user. The accuracy of recommendations is then evaluated on the test set. However, this setting does not reflect well the reality in the retail context as it is time agnostic. The availability of timestamps in the purchase records enables us to attempt a more realistic experiment.
We hence train our algorithm on past purchases and test the results on future purchases. We split the dataset according to a given point in time which acts as our "present" (the time we apply our algorithm). Purchases that happened before the split point are used for training, whereas future purchases after the split point are used for testing. Customers whose purchase histories are timestamped only after the split point are discarded. For our dataset, we choose 1st January 2019 to be the split date. More specifically, we use purchase records from January 2017 to December 2018 for training and records from January 2019 to December 2019 for testing. As it is often practiced in the recommendation literature  [8, 25] , for our experiments we discard customers who purchased fewer than 5 products in the training set.
An important aspect of our dataset and of all datasets in the retail domain is the tendency to repetitively purchase the same products at different times. It is however much more valuable for the customer and even for the retailer to recommend products that the customer has not purchased recently, or is not aware of. In addition, we noticed that if we simply randomly select N products from the purchase history of each customer as the top-N recommendations, we can reach reasonable accuracy. Thus, after several exchanges with the marketing department at TOTAL, for each test customer we decided to remove the "easy" predictions from the test set corresponding to the products that have been purchased by that customer during the training period. We also select only customers who had more than 10 purchases after removing already purchased products in the test set.
Authors consider the case of extracting and ranking temporal rules (event A→event B ) from the execution traces of programs. Each measure is evaluated in its ability to rank highly rules known from a ground truth (library specification). We observe that the measures scoring the highest are all from the groups identified in this work as G 1 and G 2 , which are also favored by our analysts. There are however some counterexamples, with measures from G 1 scoring poorly. The main difference between our work and  [14]  is the absence of a ground truth of interesting rules for our dataset. A close work to ours is Herbs  [15]. Herbs relies on a different and smaller set of measures to cluster rule rankings. Authors perform an analysis of the properties of measures, in addition to an experimental study.
The datasets used are from the health and astronomy domains. Each of them contains at most 1,728 transactions and leads to the extraction of 49 to 6,312 rules. Rankings are then compared between all pairs of measures using Kendall's τ correlation measure averaged over all datasets. The largest group of measures identified, which includes Confidence, is similar to G 1. Our use of the p-value (via Pearson's χ 2 test) in the evaluation of rule interestingness is borrowed from  [17]. In that work, the authors propose an exploration framework where rules are grouped by consequent and traversed by progressively adding items to the antecedent. The framework provides hints incrementally to help guess how each additional item would make a difference. Such a framework is suitable to some of the scenarios we consider and could be integrated in a future version of our work.
Other significant works on clustering interestingness measures include  [5, 2, 29]. In these studies, 61 measures are analyzed from both a theoretical and an empirical aspect to provide insights about the properties and behavior of the measures according to association rule ranking. The number of measures studied in these works is greater than ours. However, our work goes a step further as (1) we provide a user study performed with domain experts from TOTAL marketing department, (2) we show how association rules can be used to perform top-N recommendations, and (3) we show a comparative evaluation of the synthesized interestingness measures according to accuracy measures. An interesting research area is OLAP pattern mining, which integrates online analytical processing (OLAP) with data mining so that the mining can be performed in different portions of the database  [18, 23].
Conclusion We present our framework to enable decision support through mining, ranking, and summarization of association rules. We use large longitudinal TOTAL datasets that comprises of 30 million unique sales receipts, spanning 35 million records. In conjunction with domain expert non-scientists, we studied two scenarios: associations between a set of products and a target product, and between customer segments and product categories. Both of these scenarios led to actionable insights leading to effective decision support for the TOTAL marketers. We empirically studied 35 interestingness measures for ranking association rules and further summarize them in 5 synthesized clusters or groups. Resulting groups were then evaluated in a user study involving a data scientist and a domain expert at TOTAL.
Diversity of the CRISPR locus of Mycobacterium tuberculosis complex has been studied since 1997 for molecular epidemiology purposes. By targeting solely the 43 spacers present in the two first sequenced genomes (H37Rv and BCG), it gave a biased idea of CRISPR diversity and ignored diversity in the neighbouring cas-genes. We set up tailored pipelines to explore the diversity of CRISPR-cas locus in Short Reads. We analyzed data from a representative set of 198 clinical isolates as evidenced by wellcharacterized SNPs. We found a relatively low diversity in terms of spacers: we recovered only the 68 spacers that had been described in 2000. We found no partial or global inversions in the sequences, letting always the Direct Variant Repeats (DVR) in the same order. In contrast, we found an unexpected diversity in the form of: SNPs in spacers and in Direct Repeats, duplications of various length, and insertions at various locations of the IS6110 insertion sequence, as well as blocks of DVR deletions.
, 2014; Palittapongarnpim, et al. , 2018; Shitikov, et al. , 2017). M. tuberculosis reference clinical isolate H37Rv as well as most M. tuberculosis isolates carry a CRISPR locus together with a complete cas genes set of type III-A  (Makarova, et al. , 2015). Rare isolates lack part of CRISPR and or cas genes  (Freidlin, et al. , 2017). Partial analysis of the CRISPR diversity has been used since 1997 to explore the clinical isolates relatedness through a technique coined as « spoligotyping »  (Kamerbeek, et al. , 1997). In this technique, the presence of 43 spacers identified in H37Rv (n=35) or in M. bovis BCG (n=8) are looked for. This results in a barcode that can be easily shared and stored. Spoligotyping has led to the set-up of the first worldwide database for this pathogen counting today more than 111,000 patterns originating from 169 countries  (Couvin, et al.
Data collection One hundred ninety-eight (n=198) Sequence Reads Archives obtained by paired-end sequencing with Illumina technology were selected from a local database of more than 3,500 genome sequences based on their representativeness of M. tuberculosis lineages  (Guyeux, et al. , 2019a). Namely, the following numbers of data were included for each lineage: 55 for Lineage 1, 20 for Lineage 2, 17 from Lineage 3, 60 from Lineage 4, 25 from Lineage 5, 7 from Lineage 6, 10 from Lineage 7, 1 from M. bovis, 1 from M. caprae, 1 from M. microti, 1 from M. pinnipedii. Data were downloaded as fasta files to decrease storage space as erroneous sequence will be ignored in the analytic steps.
1. 1. 6 (ERR751749), but it can go up to three copies or down to zero (Supplementary file 2[IS6110 sheet]). No other type of insertion sequence was ever discovered inside the region (data not shown). The spacer sequences as well as those of the DR are always found in the same direction. Their order of succession is usually the expected one (the order of natural integers) although, as described below, various particular situations arise, for instance in case of duplications (Supplementary file 3). Duplications are identified not only by the order of successive spacers, but also by the relatively higher number of reads corresponding to the duplicated spacers. For instance, in an isolate belonging to L1. 1. 1. 8 (ERR718201), while most spacers were found on an average of 27 reads, spacers 14 to 21 are found in 56 reads on average, which is approximately twice as much (Figure  3 ).
Punctual variants in M. tuberculosis CRISPR Regarding intra-spacer diversity, we identified 20 spacers that harbored at least two variants, and concerned 48 (24%) out of the 198 isolates explored (Supplementary file 2[spacer sheet]). These variants consisted mainly of SNP, although a deletion was found in spacer 24 in another dataset (genome ERR702419, lineage 5, data not shown). Interestingly, some of these variants are characteristic of specific lineages. For instance, a variant of spacer 38 is found in all isolates of lineage L1. 1. 1, one mutation is found in spacer 4 in all L6 isolates to which an additional one sometimes adds resulting in two possible variants. Two variants of spacer 6 characterize the endemic Abyssinian L7 isolates (Figure  2 , Supplementary file 5). The frequency of spacer variants in L2-L3-L4-L7 was relatively low (6 independent variants detected in 107 isolates, ~5%), as compared to L1 lineage (11 independent variants out of a selection of 55 isolates, ~20%) and lineage gathering animal isolates and L5 and L6 (7 independent variants for 34 isolates, ~20%).
Around the central IS6110c, between spacers 34 and 35, the DR0 is split into two subsequences rDRa1 (upstream) and DRb1 (downstream). As expected due to IS6110 insertion characteristics, the concatenation of these two sequences is 3nt larger than DR0 since 3 additional cytosines are present at each end of the insertion  (Gonzalo-Asensio, et al. , 2018; Thierry, et al. , 1990 ). Yet, in a L5. 1 isolate (ERR702419) where IS6110c inserted downstream spacer 44, IS6110c is preceded by the first 35 nt of DR0 and followed by its 6 last nt, so that the duplicated target was this time 5nt in length (data not shown). Some variants are shared over several but not all lineages or sublineages. For instance, DR6 is found between spacers 64 and 65, in all genomes of lineages L2 to L4, and only in those; DR10 is found between spacers 67 and 68 in L6.
The CRISPR version with the two copies shares many features with that carrying the deletion, suggesting that it could correspond to its ancestral stage of evolution (Figure  4 ). The similar observation in L4. 1. 2. 1 was made independently in a study performed in Hanoi  (Maeda, et al. , 2020). These large scale deletions involved cas flanking genes in 23/198 (12%) of isolates, with two different borders in L2 isolates, two others in L4 and a third one in L3. In contrast, a single case was observed that affected Rv2813 (Supplementary file 2[IS6110 sheet]). We further explored this asymmetry using SITVIT2 2019 database (n=3852 SITs): 290 SITs harbored a deletion of spacer #1 (DVR2 in the new nomenclature) against 117 SITs with a deletion of spacer #43 (DVR65 in the new nomenclature), i. e. three times more deletions on the cas genes side.
The full dataset covered both cells that are currently forested and areas that can potentially be colonized by trees in the future (e. g. , above the current treeline). All areas currently under management (settlements, pastures) were considered to remain under the same management regime and were excluded from the simulations. The total simulated area comprised 923 cells of 1 ha size (Fig. A1 ). For all models that internally use a resolution finer than 1 ha, the environmental attributes of each 1-ha cell were assigned to all cells contained in it. This was always possible as the internal cell sizes that are being used for modeling the different ecological processes were 2 × 2 m, 10 × 10 m, 20 × 20 m, 25 × 25 m, and 100 × 100 m (cf. Table  1 ). Historic CO 2 concentrations from 1765 to 2010, as well as future concentrations from 2011 to 2100 for four Representative Concentration Pathway (RCP) scenarios (RCP2.
The spatial and temporal distribution of avalanches was estimated based on the avalanche database of the Swiss Avalanche Institute SLF, which recorded avalanches with a resolution of 1 m from 1960 to 2016 for the target landscape (Fig. A4 ). For this study, every 1-ha cell of which at least 25% was hit by an avalanche in one year was considered disturbed. Avalanche data were identical for all models, i. e. the same cells were disturbed in the same years.
Subsequently, we estimated average change rates for biomass and Shannon Diversity for the first half (2010-2050) and the second half of the century (2060-2100). Based on these values, we estimated the variance components as the proportions of the sum of squares that could be attributed to the different sources of uncertainty using the Package VCA (Variance Component Analysis; version 1. 3. 4) within the R language and environment for statistical computing  (R Core Team, 2018). In addition, we tested if the variation between models generally changed over time and with intensifying climate forcing. For this purpose, we used the results of the future projections and calculated the coefficient of variation (CV) of the total biomass and the biomass of the four most important species in each decadal time step, based on the results of all replicates of all models, for each climate scenario separately.
Past-to-present simulations Past-to-present simulationsi. e. , where forest dynamics are simulated over a long time period until quasi-equilibrium and compared with observationsare a common approach in forest landscape modeling  (He et al. , 2011). Such simulations are frequently applied for fine-tuning of model parameters and for model evaluation (e. g. ,  Temperli et al. , 2015). Here, three of the four models (iLand, TreeMig, LandClim) reproduced the observed species composition and location of the treeline fairly well. This is not surprising, as these models have been used and evaluated in the current (LandClim) or similar (TreeMig, iLand) mountain landscapes, indicating that their structures are appropriate to reproduce forest dynamics under these environmental conditions. Furthermore, fine-tuning for some species parameters was applied for TreeMig to increase consistency between data and simulations.
g. , changed spatial resolution, implementation of alternative algorithms for growth/establishment/morality, switching specific processes such as CO 2 fertilization on or off; cf. Huber et al. , 2020)  to quantify effects on simulation results. Such an analysis was, however, beyond the scope of this study. Nonetheless, we can derive cues about the main causes from our data and experiences with the models. The most obvious difference was the diverging biomass development with increasing temperature when comparing TreeMig and the other models (Fig. 2 ). For the latter, total biomass was clearly temperature limited and increasing temperatures resulted in biomass increases in grid cells below and above the current treeline (Figs. 1  and 3 ). In TreeMig, biomass was generally lower than in the other models due to a different dbh/stem-volume allometry that was introduced to better match the allometric functions applied in the Swiss national forest inventory.
Software availability LandClim is developed by the Forest Ecology Group at ETH Zürich, Switzerland. It is available online at https://ites-fe. ethz. ch/openaccess/ and can freely be used under the terms of the "GNU General Public License v3". LANDIS-II is developed by the LANDIS-II Foundation, a registered non-profit in the USA; it is open-source (Apache License 2. 0) and available online at http://www. landis-ii. org. iLand is developed and maintained at the Ecosystem Dynamics and Forest Management Group, Technical University of Munich. Source code, executable, and model documentation are available at http://iLand. boku. ac. at. The model is open source under the GNU General Public License. TreeMig is developed by the Dynamic Macroecology Group, WSL, Switzerland. It is available upon request from heike.