dataesr/scientific-paragraphs-categorization huggingface dataset:
The amount of data is sufficient for our purposes and is greater than many comparably detailed data used in the literature. Importantly, in this dataset, for each page, we could compute its most related pages. As we will see, in line with  [39] , features of related pages greatly improve the quality of predictions. On larger datasets, exact computations of most related pages are infeasible, and we will discuss in Section 8 how this can be overcome in future research. Since pages from the same top-level domain (TLD) may show a similar behaviour, we have analyzed the distribution of pages over TLD's in our dataset, see Figure  2. In our crawls, 46. 8% of the pages are from. com TLD; the other top TLDs are. org,. net,. edu, and. gov, with also a good representation of European country codes (.
Static network (SN) features measure the state of the Web graph at a given time t. Although they describe individual pages, they require an entire crawl to be computed. In this work we consider the following SN features. • Each page may be pointed to by other pages N -(p(t)), and these inlinks fall into the same two categories as the outlinks: internal and external. The counts of internal and external links per page give two SN features. The features are computed on the effective crawl (i. e. the obtained dataset and not from outside the crawl). Due to potential delays between updates and synchronisation of computation, the stored inlink information and the actual inlinks can be slightly different. • PageRank is defined as in the original work  [11] , it is a stationary distribution of a random walk with restart from a random page.
In our dataset, PageRank is updated dynamically during the crawl. More precisely, when page p is crawled, its PageRank is re-evaluated by substituting the current PageRank scores of pages that have a hyperlink to p. The current PageRank scores of pages were included in the dataset. • TrustRank is defined as in the original work  [24] , it is a stationary distribution of a random walk with restart from trusted pages. In our dataset, TrustRank is updated dynamically similarly to PageRank. The current TrustRank scores of pages were included in the dataset. • Each page may be related to other pages in term of its text content. These pages represent the part of the Web graph that is most relevant for the target page. Therefore, the SN features include the weighted averages of the static features of related pages, with weights proportional to the cosine similarity score between the semantic vectors.
Both algorithms scale better to large datasets than the standard gradient-boosting method, but are still significantly more expensive in training time than ensembles of trees. ExtraT is an ensemble of Extremely Randomized Trees (also implemented in scikit-learn  [38] ), which randomises the splitting rule at each internal node: splitting thresholds are drawn at random for each candidate feature and the best of these is selected. This lowers the variance of the model, and thus its likelihood of overfitting. Since it is an ensemble, rather than sequentially gradient-boosted learners, training ExtraT learners parallelises well, so is always the most scalable model to train on large data. HGBoost for regression and LightGBM for classification are configured with no maximum depth or maximum number of leaf nodes for the boosted trees, but instead the maximum number of samples on a leaf is tuned with cross-validation between 10 and 25 (with higher values tending to better performance).
The maximum number of iterations of the boosting process (or trees) is tuned between 200 and 500 (higher tends better). The learning rate for both is tuned between 0. 02 and 0. 1 (lower tends better). The loss function used in the boosting process is the default least squares. ExtraT for regression is also configured without a maximum depth or maximum number of leaf nodes for the trees in the ensemble, but the maximum number of samples on a leaf is tuned with cross-validation between 2 and 25 (with lower values tending to better performance). The number of trees in the ExtraT ensemble is tuned between 200 and 500 (higher tends better). For classification problems, the classes are balanced with adding balancing weights (inversely proportional to class frequencies in the data) to the training process, so the majority class need not be undersampled: all the data is kept.
Training and test datasets For each prediction problem, we form a dataset out of the predictor and target values, with one data point per Web page. The dimensionality of the semantic vector is reduced to 20 by agglomerative clustering  [38] , which groups together similar features. For each dataset, we hold out a random 25% of the Web pages as a test set. From the remaining 75% of example pages (the training set), we initially hold out one third (or 25% of the original dataset size) as a development set for tuning with cross-validation; this optimises the configuration parameters of the machine-learning models. Then, we retrain the tuned statistical model over the entire training set, and this retrained model is tested on the held-out test set. We then repeat the entire process three times.
The datasets are large, so for most prediction tasks there is little variation observed in the results of these repetitions. An exception is classification tasks with very imbalanced classes, where the repetitions help to estimate the performance metrics better. Performance metrics As performance metrics for point classification tasks, the most practically important scores are the recall (the fraction of changed pages which are predicted correctly), and the F1-score (a mean between recall and precision: the fraction of pages predicted to change which actually do). We also report precision, and the balanced accuracy score (the average of the recall scores per class). For point regression tasks, we show the coefficient of determination R 2 , the mean absolute error (MAE), and the median absolute error (MedAE).
MedAE is provided because it is more robust to outliers than MAE. The R 2 score is the proportion of variance of the target variable that is explained by the independent predictors in the model, with the best possible score 1, and a score of 0 for a constant model that always predicts the expected value of the target, disregarding the input features; negative scores are possible for arbitrarily worse models. Note that since the variance of the target is dataset dependent, R 2 is not comparable across datasets, so across prediction tasks for different target variables. For estimating the importance of individual features for the model learnt, we use permutation importance  [10] , a model-agnostic metric based on comparing the performance of the model trained with all features with that of the model trained with the feature of interest shuffled among the samples in the dataset, and all other features intact.
Both LightGBM and ExtraT classifiers are able to handle the imbalance datasets, and scaled well. However, ExtraT shows better prediction performance than LightGBM, thus we only report the results of the ExtraT classifier. The models are trained with increasingly complex feature sets and increasing history size. In other words, for the history size of one, just one week of dynamic features has been included and for the history size of eight, the feature set contains all dynamic features. The target score for tuning the   As can be seen from our initial statistical analysis (see Figure  3 ) most pages retain their class in subsequent weeks. This motivates us to compare the results to a very simple baseline that predicts the same number of new outlinks as in the previous week (NNL-Pr). Thus, in week 9 NNL-Pr predicts the same class as in week 8.
Number of new links (NNL) The last row of Figure  16  shows Spearman rho for the NNL target. As we see, the NL-ET and NNL-ET are the most correlated rankings with NNL for both internal and external outlinks. Again, the average resolution of ties explains the high Spearman's rho between NNL and NL-ET. An interesting point in Figure  16  is that the NNL-Av ranking is a good representation of the NNL ranking. It means that if we have the historical data of the pages, we can simply follow the ranking obtained by averaging the number of new outlinks in previous weeks to predict the NNL ranking. Figure  19  depicts Precision@k% performance of different rankings compared to the NNL target ranking for different values of k. NNL-ET again produces the best ranking, we observe that it has the highest values for almost any k.
In particular, NNL-ET can detect the hottest pages in the leftmost region of the charts better than other rankings. Again, since random resolution of ties is used to compute Precision@k%, the right region of the former tends to look like the random ranking. This is more visible for external outlinks because this data contains more zeros. It should be taken into account that although training the regressor for NNL-ET might be a time-consuming process, once the model is constructed, it can be used to generate the estimation of NNL in a very short time.
Conclusions and further research In this work, we have extensively analyzed the problem of predicting emergence of new outlinks on a Web page using the data of ten weekly crawls. We have established that there is a significant difference between internal and external outlinks, thus, we have considered them separately. Statistical analysis reveals that most Web pages have no new outlinks in one week. Among the pages that have new outlinks, there is a high variability in the number new outlinks, both internal and external. Nevertheless, there is a considerable stability in the distribution of the new outlinks over pages from one time period to another. This observation motivates a very simple but informative prediction heuristic, NNL-Pr, that predicts the same number of new outlinks in the next period as in the previous period.
On the other hand, the content change rate is not very informative for emergence of new outlinks. In general, we recommend to design prediction methods separately for the content and the link change rate. In our work, we used cosine distance as content similarity metric for computing related pages. Although the cosine distance is more efficiently calculated than the temporal content similarity metrics as in  [39] , the nearest neighbor search is still computationally expensive, and can jeopardize the scalability of our methods. This can be potentially resolved by, for example, using GPU as in recent work  [26] , or parallel implementation as proposed e. g. in  [17]. Both approaches are promising for scaling our methods to larger datasets. In the future, we would like to experiment with different temporal similarity metrics.
For example, Dynamic Time Warping (DTW)  [9] , a dynamic programming algorithm that finds patterns in time series data, that has been used in  [39]. DTW has been shown superior to other similarity measures but it has prohibitive computational complexity on large datasets. A promising recent development is the Scalable Warping Aware Matrix Profile (SWAMP) introduced by Alaee et al. [3] , the exact algorithm for DTW motif discovery in massive datasets. Using these new computational algorithms, DTW can be introduced in our methods and potentially improve predictions.
Measurements of page change Many hints for features come from Web measurement studies, which observe correlations between page features and page change. Over a crawl of 150 mil. pages once a week for 11 weeks, with content page similarity measured as the overlap in word n-grams (shingles) between the page variants, good predictors of page change are: the degree of previous content change, the length of the page (longer pages change at a higher rate), and the top-level domain of the page. For example, the rate of page text change for. edu pages was below 10%, but that of. com pages was almost 30%  [21]. Additionally, they found that keyword-spam pages have a particularly high rate of change. Adar et al. [2]  analyzed a crawl of 55,000 Web pages. Data was collected from 612,000 English-speaking users of a live search toolbar over a period of 5 weeks (so biased in favor of pages that do change, since they were chosen by humans to visit).
They collected data daily for 30 days on pages in two distinct topics (Ebola and movies), for which they considered the same 22,200 and 27,353 pages each day per topic respectively. They find that page topic is an important predictor for change frequency and for the expected number of new links. They also find that revisiting already downloaded pages leads to new pages, but those pages don't necessarily have the same topic. For both topics almost all pages either have a change frequency in the 0 to 0. 1 or 0. 9 to 1. 0 range, so either very often or not at all. This work is limited in the fact that they only studied two topics, and their definition of page change might be somewhat loose in the sense that it does not account for dynamic html content such as ads. Ntoulas et al. [35]  have crawled weekly snapshots of 154 websites over the course of a whole year, where they downloaded up to 200,000 pages per site each week.
Unlike in later studies  [2, 40] , the URL depth was not found important. A pruned decision tree was the best classifier using static page features, and when historic change data is also available, it can refine the decision of the classifier by moving the page to another class. The error rate is 15%; the relative feature importance values are not provided. Specific focus is given to predicting, still using mostly static page features, the number of in-links to pages  [37] , as a sign of page status on the Web. The ground truth is obtained by a Google Web Search. The features are diverse: the number of words, the URL depth, the scope of the topics covered, the number of out-links, the textual cosine similarity between two snapshots of the page, and the popularity (traffic) on the entire domain; all appear important to the prediction.
A nonlinear regression using model trees (algorithm M5') performed best, however, this predicts (across a dataset of 82,068 pages) a number of in-links that has a correlation coefficient with the ground truth below 0. 7. Clustering 300,000 pages (obtained from the WebArchive 1  ) across 210 websites (collected from the Internet Archive 2  over one year) by combining static and dynamic page features leads to clusters with similar page change patterns  [44]  (the precise type of page change is not specified). Some static page features are content-related: features computed out of the term frequency-inverse document frequency (TF-IDF) statistics of the 1000 most frequently appearing words in both the page and the URL, the number of images, tables, or words, the file size and type (HTML, plain text).
The target to be predicted is the categorical frequency of change, with five such categories defined. The dataset consists of only 3,122 pages with different changing rates, crawled over 12 weeks; neither the domains of the pages, nor the class sizes are specified. The prediction is done by a modified Random Forest classifier: 87. 98% of the pages in the test set of 624 were classified correctly. This method has a serious shortcoming: acquiring the features of a page via frequent crawling is an extremely expensive process, which explains the limited testing.
However, a short report published in 1876  [3]  demonstrated that a great portion of empirical knowledge accumulated over centuries in Asia was not fully valorized between these two dates. Nevertheless, its potential anticancer activity was originally demonstrated against the development of tumors in animals  [4]. The relief produced by a topical application of curcumin ointment in patients with external cancerous lesions was also revealed  [5]. At the beginning of 2023, the number of publications related to the use of turmeric's main phytochemical as an anticancer agent reached 7267 in the PubMed database. Although curcumin represents "the leading" molecule, a significant part of these articles refers to 2 of 31 "curcuminoids"  [6]. The term also covers other plant secondary-metabolites belonging to the diarylheptanoids class  [7] , semi-synthetic structures such as prodrugs consisting of promoieties attached to the phenolic hydroxyl groups  [8]  and synthetic chemical molecules  [9]  that were initially produced for optimizing pharmacological potency  [10].
Sampling to estimate prevalence First, we assessed the prevalence of C. botulinum in birds at the four sites, annually from 2018 to 2020, through sampling of hunted birds at site A, and through cloacal swab sampling at sites B, C and D. At sites B, C and D, the sampling strategy was opportunistic: we took advantage of capture and banding operations that already took place (generally between September and March, using traps such as cages or nets) to carry out cloacal swabbing on banded birds. All cloacal swabs were realized by trained and qualified banders. Swabbing was mostly carried out on two species, mallards and common teals, because these were the most frequently captured and banded birds, and because they are both of interest for the study of botulism (see 2. 3 Data processing). In total, 171 banded birds were swabbed: 42 at Baie de l'Aiguillon (B), 23 at Grand'Mare (C), and 106 at Lac du Der (D).
Data processing We pooled common teals and mallards for analyses (i. e. , we did not distinguish between the two species) as they share many biological and ecological characteristics: both are dabbling ducks (Anatinae) and are susceptible to botulism, they share the same mostly granivorous diet during winter, and are both abundant in France and in the three study areas. Pooling the two species also made it possible to work with a larger dataset. Ultimately, the swabbing dataset consisted of a total of 171 swabbed banded birds for sites B, C and D, and 29 hunted birds for site A (also common teals and mallards). To evaluate the dynamics of infection in the studied population at sites B, C and D, we estimated the prevalence at each site during each period of interest, defined as periods of the year with weather conditions as homogeneous as possible between the different sites and years (mean temperatures, humidity, and rainfall).
Exposure of birds to C. botulinum is known to remain constant during consistent weather conditions. Since October did not have consistent meteorological conditions, we did not estimate prevalence during this month of the year. Indeed, we wanted to focus on sure and consistent periods from year to year, and after performing a PCA of three weather variables (temperature, precipitation, humidity), October was not constant among 2018, 2019 and 2020. At sites B, C and D, we therefore estimated botulism prevalence during the periods August/September, November/December and January/ February, depending on the availability of data for each year. At site A, the available data provided only a snapshot of the prevalence (as they were collected over only one day), and it was not possible to study the dynamics of the outbreak.
, 2014). We were therefore able to estimate the limits of an 80% credible interval on the prevalence (interval with 80% chance of including the true value of the prevalence) with the function qbeta from the R software. For site A, we estimated C. botulinum prevalence in November 2018 by evaluating detection in ceca, but also separately for four other organs (intestinal contents, liver, gizzard, and keel muscle), to assess C. botulinum tropism. All statistical analyses were performed with R 4. 0. 3 and the interface Rstudio version 1. 1. 463.
Acknowledgments We would like to thank:  Natacha Piffeteau  ( Departmental Federation of Hunters of the Eure -FDC27 ),  Emmanuel Joyeux  and  Sylvain Haie  ( OFB ) for their valuable help with banding and counting data, and other technical activities, and the  Departmental laboratory of analyses of the Ain and Aube  ( LDA 01  and  LDA 10 ) for their support in the autopsies and analyses of birds. We also thank all members of the  Departmental Federation of Hunters of the Marne  ( FDC51 ) and the  Departmental Service of the Marne  ( OFB -SD51 ) for their contribution to data collection in this study.
Most of optical methods for the identi cation of microorganisms rely either on vibrational spectroscopies, such as Raman and infrared spectroscopy, or on morphotype imaging within the visible range. With the most recent developments of quantum cascade lasers, a fast imaging system able to collect both of these information, vibrational (composition-based) and morphotype, is now within reach. However, the potential of such system to be more precise than conventional imaging or spectroscopy has yet to be demonstrated. Here, we report a practical protocol for microbial colonies identi cation by means of QCL-based multispectral infrared imaging, as well as the acquisition of a rst database for performance evaluation. To this end, optical ngerprints of 2253 colonies belonging to 8 microbial species were recorded at 9 different wavelengths in the mid-infrared range.
Multispectral acquisitions After culture, alumina membranes are dried for 15 minutes, then deposited on a microscopy stage, 2 mm above sensor without intermediate optical element. Each region of interest is illuminated with a sequence of nine successive wavelengths (Fig. 3 ). An image is recorded for each wavelength. Operations are controlled by a custom Python 2. 7 script ensuring communication between the laser source and the sensor. For each wavelength, the laser is set to its maximal power and the sensor parameters adjusted in consequence. This ensures the highest signal-to-noise ratio for all conditions. The resulting stack of images presents 3D optical patterns which encompass both spectral and morphological information.
Database composition To test the performances of multispectral mid-infrared imaging as an identi cation technique, a database containing 2253 individual colonies from 8 microbial species and 3 strains of S. epidermidis was acquired (Table  3 ). This database was designed to include very diverse micro-organisms, for instance the yeast Candida albicans versus bacteria, or the Gram-positive Listeria innocua versus Staphylococcus species, but also phylogenetically-close organisms, such as the two Enterobacterales (Enterobacter cloacae and Escherichia coli) or the three Staphylococcus subspecies. Machine learning classi cation was nally performed with strati ed data and a tenfold cross-validation test  [42]. Nine of these groups were used to train the algorithm, while the last one was kept to test the resulting model.
Both species-level and strain-level classi cation yield promising results, with average CIRs of 94. 1% and 94. 3%, respectively. However, this slight increase of performance in strain-level classi cation compared to species-level can be counterintuitive. In most cases, subdividing a class (here a species: S. epidermidis) into sub-classes (strains) leads to performance loss, since it increases the number of classes and diminish the number of examples per class. Here, the opposite is observed. One possible explanation is that, for species-level classi cation, S. epidermidis is overrepresented in the database, which might lead to confusions. Moreover, the three S. epidermidis strains might possess largely different morpho-chemical ngerprints. This tends to be con rmed by intra-species confusion (0-2.