dataesr/scientific-paragraphs-categorization huggingface dataset:
(6)  showed that a new algorithm using deep learning performed well on a research dataset, especially when retrained, and that more traditional algorithms worked best on a clinical dataset. Finally, Khademi et al. (12)  observed better performances for new algorithms compared to partial volume average modeling or LST LPA algorithms. In general, it thus appears that algorithms using deep learning methods may outperform more traditional tools. Notably though, existing studies have often used the default parameters for all tools, which can obscure the real potential of the segmentation methods  (6, 11). Moreover, the description of the manual WMH segmentation is often scarce, without reference to guidelines, thus impeding accessibility and reproducibility of findings  (15). The objective of the present study was to provide evidence on broadly applicable tools to automatically segment WMH in a multicenter context on cognitive impairment and dementia.
No other preprocessing was included. The model weights used for pgs were the same as used during the MICCAI challenge [see Kuijf et al. (10)  for more details]. For sysu_media, only FLAIR image were mandatory as an input. sysu_media also required a training dataset. The algorithm was thus tested with two variations: either with the default training dataset from MICCAI challenge as for pgs, or with the training subset based on DELCODE data (see above). For the first one, no additional preprocessing was performed. To retrain the sysu_media UNet, all brain images from both DELCODE training and test subsets have been brain-extracted and lightly bias corrected using FSL BET (20) (option -f. 2 and -B) and, then noise corrected using the spatial adaptive non-local means filtering  (21)  implemented in the Computational Anatomy Toolbox (CAT12.
Volume-based results An overview of the results is provided in Table  3  and illustrated in Figure  1. The average similarity coefficient (DSC) was moderate for all segmentation tools with values ranging from 0. 602 (± 0. 169 SD) for LST LGA to 0. 702 (± 0. 109) for sysu_media retrained on the DELCODE subset. The sensitivity was moderate with LST LPA performing the best with 0. 714 (± 0. 190) followed by sysu_media retrained on the DELCODE subset, while LST LGA had the lowest performance with 0. 562 (± 0. 200). The precision was moderate to good with the best performances for pgs with 0. 768 (± 0. 151) closely followed by sysu_media retrained on the DELCODE subset with 0. 741 (± 0. 190) and BIANCA with 0. 731 (± 0. 179) with the lowest performance measured for sysu_media trained on the MICCAI dataset with 0.
Number of lesions The average global score (F1-score) was moderate with best performances for sysu_media retrained on the DELCODE subset with a mean F1-score of 0. 642 (± 0. 109 SD) closely followed by pgs with 0. 635 (± 0. 162) and the lowest performance for LST LGA with 0. 466 (± 0. 113). The average sensitivity to detect lesions of all methods was poor to moderate, with value ranging from 0. 372 (± 0. 112) for LST LGA to 0. 631 (± 0. 187) for sysu_media trained on the MICCAI dataset, closely followed by pgs with 0. 624 (± 0. 182). The average precision was generally good or moderate for all methods with best performance for sysu_media trained on the DELCODE subset with 0. 761 (± 0. 145) closely followed by LST LPA with FLAIR only and 0. 753 (± 0. 198) and the lowest performance measured for sysu_media trained on the MICCAI dataset with 0.
65), except for the insular and temporal cortices. The sysu_media trained on the MICCAI dataset and pgs had poorest global performances in the corpus callosum compared to the same measures in the other ROIs or at the whole brain level, with sensitivity and DSC values below 0. 5, but good precision. Inversely, these algorithms had best performances in the insular cortex with all measures above 0. 65. All results presented in this section are summarized in Supplementary material.
High vs. low global WMH burden At volume level, the average DSC, sensitivities and precisions were generally higher in the high WMH load group than in the low WMH load group for all methods, with a difference of at least 0. 1 reaching more than 0. 3 for all indices in sysu_media trained on the MICCAI dataset. When looking at the level of lesions, the observations were more heterogeneous. Except for pgs and sysu_media trained on the MICCAI dataset, where we observed a notable increase in DSC, sensitivity and precision in the high WMH load group compared to the low WMH load group, all other algorithms showed either similar results between the two subgroups or a decrease of all performance metrics in the high WMH load group vs. the low WMH load group. All results presented in this section are displayed in Table  4.
After discussing advantages and challenges of the evaluated tools, we provide recommendations for future studies including automated WMH detection in a multicenter context. A comparison with the past literature on the evaluation of automatic WMH segmentation tools is complex due to multiple factors, including differences in algorithms being evaluated, disparate data (in terms of quantity and quality) used for evaluation, parameters of the tools tuned or not, and heterogeneous evaluation measures being reported  (27). In this study, we thus computed performance measures already implemented in previous studies  (6, 11, 12)  to enhance comparison across studies and to get simple but efficient performance indices widely used in clinical and research practice. Thus, it is to note first that the initial performance estimations of tools are often evaluated in a single center, with tuned parameters leading to better performances, and thus cannot be generalized to other datasets.
For example, the high performances reported in the original paper of LST LGA  (7)  were not replicated in a series of comparative studies  (6, 11). Of note, LST LGA was developed in the context of multiple sclerosis, not for WMH of presumed vascular origin. Even if WMH are visually similar, underlying pathologies are not the same and anatomical location may differ, especially for small lesions. In general, our results are consistent with previous studies using similar approaches. Indeed, Heinen et al. (11)  observed best performances for kNN-TMP and to a lesser extent for LST LPA (with FLAIR only) over other methods such as LST LGA. Unfortunately, no algorithm using deep learning was evaluated in this paper. Another previous study (6) compared traditional and deep learning algorithms and obtained best performances for the deep learning methods [NicMSlesion  (28) ] and the LST LPA, depending on the dataset that was assessed (resp.
research or clinical setting). Interestingly, for the deep learning algorithm, the authors observed significantly better performances when retraining was performed using a subset of the evaluated dataset, compared with the default training dataset. We observed the same with our evaluation. This observation is not surprising, but it highlights the importance of the retraining approach (even with a small sample, 20 images here) on study datasets to achieve optimized segmentation results for deep learning algorithms. Moreover, Vanderbecq et al. (6)  showed that with the deep learning tool, the variability in the WMH segmentations among participants could be large (for the research dataset) or narrow (for the clinical dataset). In our comparison, the variability of our performance measures was the smallest for sysu_media, retrained on the DELCODE subset.
As expected from our results for global WMH volumes, the performances across the different tools for the detection of the total number of lesions were moderate. Overall, sysu_media retrained on the DELCODE subset performed the  best, closely followed by pgs. In both cases, a high precision was coupled with lower sensitivity, a behavior that was found for all algorithms but sysu_media (trained on the MICCAI dataset). This pattern reflects the capacity of these algorithms to detect fewer lesions than expected, resulting in an underestimation of the lesions. Overall, both precision and sensitivity have to be evaluated in line with all parameters to have an impression of the overall performance of each algorithm. Second, we also found that the location of the lesion is important, especially because of the underlying nature of the tissue.
The initial parameters of the tools (such as the kappa threshold for LST LGA) but also the threshold of the output probability maps should also be tuned according to the studied dataset. Indeed, default values of tools should be adapted to better fit own specifications that may vary from the original publication. Finally, the model parameters should be optimized by retraining when using deep learning algorithms to improve the WMH segmentation results as shown above. A few limitations to our study need to be reported. First, we used only one trained rater for the RS. Some studies have shown that the inter-rater reliability may be low for WMH detection  (34). However, this problem was mostly found in studies published before the consensus definitions of WMH as described by the STRIVE criteria (1).
Indeed, more recent studies tend to have a reduced variability between and/or within raters [inter/intra-rater ICC > 0. 97  (6, 11) ]. This led us to the decision that one trained rater with quality assurance using a step-by-step manual segmentation protocol and continuous monitoring by experts would be sufficient in our study. Second, we acknowledge that the number of participants in our dataset may be low. The limitation here was notably due to the time required to segment each FLAIR image manually, which was more than 8 h for those participants with heavy WMH burden. We thus wanted to focus more on a qualitative dataset, extremely carefully segmented and documented, than on a quantitative dataset realized quickly at the expense of good quality. At the end, using only 2 participants per DELCODE center for the training subset, we show that the deep learning method sysu_media performed best and with the lowest variability for most of the measures.
Hence, the size of a given training dataset may not be such an issue compared to the importance of giving the algorithm a chance to adapt to the properties of the sample and desired segmentation performance. Nevertheless, the quality of segmentation per scanner could not be evaluated with such small groups. Furthermore, all images were solely acquired on Siemens scanners using a harmonized protocol. Thus, we acknowledge that our results may not be replicable with other machines and differing protocols between images. Of note, a previous study showed that performances of LST-LGA and Bianca (and LST-LPA to a lesser extent) algorithms were particularly robust to different scanners  (6). We also decided not to apply final masking to exclude regions excluded in the manual segmentation (see "Reference segmentation procedure" section).
We acknowledge that this decision might unfairly bias the results in favor of methods with a retrained dataset. Finally, the choice of measures for the evaluation of performances is always a matter of debate. Some efforts have been made to propose guidelines with metrics suitable for the evaluation of segmentation  (35). But in practice, it is always difficult to use multiple results in a single paper without overwhelming readers with too much information, especially when the evaluation is not only globally, but at multiple levels, as in our study. We thus favored to use simple reliable metrics also used in many other comparative studies and applied these metrics not only at the whole brain voxel level, but also in relation to the total number of WMH lesions and their location. To summarize, we showed that deep learning algorithms, especially when retrained, performed the best in the present multicenter context of cognitive impairment and AD dementia.
Acknowledgments We are grateful for the tremendous efforts of all DELCODE study teams across participating DZNE sites. Further, we express our sincere gratitude to all volunteers and their family members that participate in the DELCODE study. We gratefully acknowledge all administrative and scientists staff members involved in data acquisition, data management, as well as quality control. We thank  Hongwei Li  for his help to use sysu_media and all members of  Wirth lab  for their help at various stages of this study.
Data availability statement The data that support findings of the present study are available on reasonable request from the DELCODE database. Requests to access these datasets should be directed to the German Center for Neurodegenerative Diseases [Deutsches Zentrum für Neurodegenerative Erkrankungen e. V. (DZNE)], Bonn. Respective scripts for data analysis are available from the corresponding author (MG) on reasonable request. The adapted sysu_media implementation and DELCODE model weights are available as an open-source package on GitHub (https://github. com/0rC0/WMHpypes).
e. , the effective number of layers index, "ENL," see below) and plot species richness (see Appendix S2 for more details). Spatiotemporal trends for the whole experiment were estimated using Gaussian radial basis functions (functions "auto-basis," "eval_basis" from the R package FRK, see Appendix S2: Section S1 and  Wikle et al. , 2019). Our model fits explained more than 90% of the loggers' temperature measurement variability. The fitted models were used to predict daily minimum, mean, and maximum temperature for all experimental plots with a standard error from 0 to 2 C during our sampling period (see Appendix S2 for the complete procedure).
Root traits Root functional traits were measured from BEF-China Site A from September to October 2013 using two to three tree individuals per species per diversity level. First-order roots were collected, cleaned, scanned, and analyzed by WinRHIZO (Regent Software, Canada). After measurements, roots were air dried at 60 C for 2 days and weighed. Average RD (in mm) and SRL (in m g -1 ) were calculated from the measurements of each species at all species richness levels  (Bu et al. , 2017). The mycorrhizal status of the tree species was determined from the literature  (Haug et al. , 1994; Hawley & Dames, 2004; Wang & Qiu, 2006). The mycorrhizal status was assumed from the literature and confirmed by recent measurements in the same experiment  (Singavarapu et al. , 2021). However, intraspecific root functional trait variability can be high and may change over the course of an experiment, depending on the biotic context (e.
For each plot, the ENL was computed. First, the scans were filtered using a statistical outlier removal filter (SOR, N = 10, SD = 3) in CloudCompare 2. 9. 1 software. Taking into account the dimensions of each plot (~667 m 2 ), each point cloud was clipped in a 20-m 2 around the scan center (~400 m 2 ). The point clouds were voxelized into a voxel grid of 5 cm voxels using the R package VoxR  (Lecigne et al. , 2018). Then, they were grouped in vertical slices of 50 cm and, for each slice, we quantified the proportion of filled voxels. The ENL was the result of calculating the inverse Simpson Index: ENL ¼ 1= P n i¼1 p 2 i , where n refers to the number of slices, calculated as (height maxheight min )/50 cm; and p i is the proportion of filled voxels of the ith slice.
Statistical analyses A description of all the variables used in this study can be found in Appendix S5: Table  S1. All data handling and statistical calculations were performed using the R statistical software version 3. 6. 1. All R scripts used for this project can be found in our Zenodo release (https:// doi. org/10. 5281/zenodo. 7225739). In order to avoid any deviation due to scale differences between variables, all explanatory variables were centered and divided by two standard deviations for our analyses using the R "rescale" function from the arm package. Collinearity of root trait indices was inspected by Pearson's correlation (Appendix S6); highly correlated variables were excluded by our both-way step selection algorithm favoring the simplest model  (Venables & Ripley, 2002) , that is, the R "step" function from the stat package.
Our SEM was fitted using the R "sem" function from the lavaan package  (Rosseel, 2012). The quality of our model fit on the data was estimated using three complementary indices: (1) the root-mean-squared error of approximation (RMSEA), (2) the comparative fit index (CFI), and (3) the standardized root-mean-squared residuals (SRMR). A model fit was considered acceptable when RMSEA < 0. 10, CFI > 0. 9, and SRMR < 0. 08.
ACKNOWLEDGMENTS We gratefully acknowledge funding by the  Deutsche Forschungsgemeinschaft (DFG ;  German Research Foundation   -319936945/GRK2324 ) and the  University of Chinese Academy Sciences (UCAS). We acknowledge the support of the  TreeDì research group , especially many local helpers involved in collecting the samples. We also thank the laboratory members of the  Experimental Interaction Ecology group  for their support, especially Alfred Lochner,  Anja Zeuner ,  Alla Kavtea , and  Linnea Smith  for their help during the laboratory measurements. The  Experimental Interaction Ecology group  is supported by the  German Centre for Integrative Biodiversity Research (iDiv). We gratefully acknowledge the support of iDiv funded by the  German Research Foundation  ( DFG-FZT 118 ,  202548816 ).
The recommendation is based on a similarity between the user-provided settings and a large set of precomputed experiments. AdvisIL makes class-incremental learning easier, since users do not need to run cumbersome experiments to design their system. We evaluate our method on four datasets under six incremental settings and three deep model sizes. We compare six algorithms and three deep neural architectures. Results show that Ad-visIL has better overall performance than any of the individual combinations of a learning algorithm and a neural architecture. AdvisIL's code is available at https: //github. com/EvaJF/AdvisIL.
It prevents the users from benchmarking each CIL algorithm and backbone network, hence saving them time and computation efforts. AdvisIL is assessed by an evaluation protocol which uses four test datasets and eighteen scenarios, allowing us to highlight the relevance of our recommendations in a variety of experimental settings. To allow the use and the enrichment of AdvisIL by the community, and thus the quality of its recommendations, we will share the code and the precomputed experiments on which the method is based.
Defining a CIL process The objective of CIL is to train a model that integrates all classes of a dataset that arrives in a stream. Without loss of generality, we assume that examples of past classes cannot be stored (exemplar-free setting), and that the model relies on a convolutional backbone network b. We consider a sequential learning process composed of k non-overlapping steps s 1 , s 2 ,. , s k. A step s i consists in learning from the examples contained in a dataset D i. To each dataset D i is associated a set P i of classes, such that each learning example in D i uses a class belonging to P i. The datasets composing the complete dataset D = D 1 ∪ D 2 ∪ • • • ∪ D k satisfy the following constraint: for i, j ∈ {1, 2,. , k} with i ̸ = j, P i ∩ P j = ∅. The number of classes of P 1 , which corresponds to the initial step, is denoted α.
All these parameters form a configuration. Definition 2 A configuration is defined as an 7-uplet (a, b, D, m, k, α, β) containing: the name of an algorithm a, the name of a backbone network b, a complete dataset D, the values corresponding to a memory budget m, a number of steps k, a number of initial classes α and an incremental update size β. The training of this model is performed as follows. At the first step s 1 , the model M 1 is trained on the dataset D 1 that involves α classes. For each of the next steps, the same procedure is applied. For i = 2, 3,. , k, at the step s i , the model M i first recovers the weights from the model M i-1 that we obtained in the previous step. It is then trained using the examples of the dataset D i involving β new classes. The overall classification performance r on D of the final model M k is called the average incremental accuracy and is computed by: r = 1 k -1 k i=2 q i , (1) where q i is the accuracy of the model M i on D 1 ∪ D 2 ∪ • • • ∪ D i ,.
Step 2. In this step, we rely on the reference configurations in C whose scenario is v. For each dataset D i ∈ J , we construct the ranking of all pairs (a, b) ∈ A × B according to their classification performance on that dataset with the scenario v. For i = 1, 2,. , l, the ranking is defined by a partially-ordered set Rank i = (A × B, ⪯) with the following order relation. For a, a ′ ∈ A and b, b ′ ∈ B, we have: (a, b) ⪯ (a ′ , b ′ ) iff r ≥ r ′ , (4) where r and r ′ are the classification performances of the experiments which use the reference configurations (a, b, D i , m, k, α, β) and (a ′ , b ′ , D i , m, k, α, β) of C, respectively, and are computed using Equation 1. To the ranking Rank i of the dataset D i , we associate a function that gives the position of a pair (a, b) in it. For i = 1, 2,.
, l, this function is: g i : A × B → {1, 2,. , card(A × B)}. Finally, for a pair (a, b), we can aggregate all its positions in the rankings associated to the l datasets by: Agg : A × B →{l, l + 1,. , l • card(A × B)} : (a, b) → l i=1 g i (a, b). (5) The recommendation provided to the user is the pair(s) of algorithm and backbone network with the lowest overall position computed by the function Agg.
Experiments 4. 1. Generating the reference configurations We describe the CIL algorithms, backbone networks, datasets and scenarios used to generate the set of reference configurations. EFCIL algorithms. In our experiments, we use a representative panel of the algorithms presented in Section 2: LUCIR  [19] , SPB  [52] , SIW  [4] , DeeSIL  [2] , DSLDA  [15]  and FeTrIL. We remind that LUCIR, SPB and SIW update the backbone network at each incremental step, and thus focus on the plasticity of representations. In contrast, DeeSIL, DSLDA and FeTrIL use a representation which is fixed after the initial step, and thus favour stability. We implement all algorithms using PyTorch  [37]  (see the supplementary material for implementation details). Backbone networks. In our experiments, we use: ResNet18  [17] , MobileNetv2  [43]  and ShuffleNetv2  [27].
ResNet18 is widely used in the CIL literature. MobileNetv2 and ShuffleNetv2 are designed for high accuracy while considering computational efficiency for embedded applications. These backbone networks are scaled to fit various memory budgets using Equation  2. Datasets for reference configurations. We consider five datasets which are sampled from ImageNet  [11]  and denoted by INFood, INFauna, INFlora, INRand 0 and INRand 1. The first three datasets are thematic, and were obtained by sampling leaf ImageNet classes which belong to the "food", "fauna", and "flora" sub-hierarchies, respectively. The two other datasets were obtained by randomly sampling classes from ImageNet. Each dataset contains 100 classes, with 340 images per class for training, and 60 images for testing. Each sampled class is only used in one dataset.
Since we have performed many experiments to test the method, the datasets are designed in such a way that the total execution time remains tractable. CIL scenarios. The memory budget m is defined as the number of parameters of the final model (that contains 100 classes here) and is taken in {1. 5M, 3M, 6M}. The chosen budgets reflect the computational constraints of embedded devices, for which CIL is particularly useful  [16]. The other parameters of the scenarios are summarized in the first row of Table  1. Following  [42] , half of the scenarios have an even split of classes among all states. The other half starts with a higher number of classes in the initial step and evenly splits the remaining classes among subsequent steps, as proposed in  [19]. In this way, we cover the two types of scenarios most frequently used in literature.