dataesr/scientific-paragraphs-categorization huggingface dataset:
Nevertheless, one might want to use undersampling algorithms without reaching a 50-50 balance rate. This has no impact on the rest of the paper. In  [8] , the authors showed that the output score of the model trained on a rebalanced dataset is biased (because the prior has changed) and can be readjusted. By denoting s the output score and s adj the output score readjusted to the original data distribution, we have: s adj = βs βs -s + 1 (5) where β = |D + tr | |D - tr | in the case of balancing an unbalanced problem. Readjusting the output to the initial prior naturally reduces the calibration errors but, at this point, we have just corrected the bias brought by undersampling learning and did not correct miscalibration due to the architecture of supervised classification models.
Measure of global calibration in imbalanced classification Another solution for ensuring the minority class is well-represented in the computation of the calibration error would be to change the weights of the bins in ECE by creating a smoothing function. To do so, the function that we propose as a contribution must: 1. Maintains the order of the weights. 2. Reduce or conserve the relative distance between the weights. 3. Have a lower impact for more balanced datasets. From 3, we can deduce that we must use a measure of the dataset imbalance. The measure we chose is the size of the smaller class. The most straightforward method that would respect all the conditions would be to use a function of the fraction of the minority class as a power of the number of observations in a bin. We can think about different variants: Let denotes γ = N + N ∈ [0, 5].
• α strong = 2γ: strong re-weighting compared to ECE, close to an equally weighted average in case of highly imbalanced datasets as it gets closer to 0. • α soft = 0. 5 + γ: more conservative, will conserve quite the same weights as in ECE. • α smooth = -γ ln(γ)+(1-γ) ln(1-γ) ln(2) : In terms of aggressiveness in reweighting, it is in the middle of the two latest proposed α. As a result, the formula of the reweighted ECE, ICE (Imbalanced Calibration Error), would be: ICE strength = 1 Z M m=1 |B m | α strength frac pos (B m ) -conf(B m ) (8) where Z = M m=1 |B m | α strength is a normalization constant and strength ∈ {soft, smooth, strong}. The first two propositions of α strength are direct linear transformations of the size of the minority class. These approaches offer a quick solution to the problem but are hard to generalize for datasets with more than two classes.
The last proposition uses the Shannon entropy  [26]  to generate α smooth. The entropy coefficient for a dataset with two classes is between 0 and ln(2) so by dividing it by ln(2) we will obtain an α between 0 and 1. The advantage is that it can be extended to allow the calculation of ICE for multi-class classification problems; thus a more general form would be: α smooth = - |Y| k=1 γ k ln(γ k ) ln(k) where |Y| is the number of class and γ k denotes the share of the k-th class. If the dataset is perfectly balanced, then γ = 0. 5 ⇒ α strength = 1 and the formula (8) reverts to the original ECE formula (3). Even though these calibration metrics can capture a calibration measure along the entire confidence score axis, it differs from the attempt of estimating eq. (  2 ) (whereas the ECE does) because of the re-weighting.
New proposed parametric calibration methods In the previous section, we addressed the calibration problem, showing that we needed to be in the correct probability distribution to fit parametric calibration mappings, which were designed to perform well on such spaces (as Platt Scaling with the sigmoid shape). However, in this section, we propose three new parametric methods designed to deal with the biased probability distribution so that we can apply a single transformation to recalibrate the model. As we can see in Figure  6 , the output score of a model trained on rebalanced data is far too high compared to the observed likelihood, which is normal because the model returns scores related to a balanced prior distribution. As expected, the more the dataset is imbalanced, the more the output is biased.
Datasets and experimental methodology To test the new calibration methods, we selected 6 publicly available well-known datasets. We arranged each one to fit into a binary imbalanced dataset with a minority class accounting for around 1% or 10% of the total number of samples. For instance, for a multi-class dataset, we select class 0 versus the others and remove positive or negative samples to reach the fraction of positive desired; see Algorithm 1. For some datasets, we chose different proportions of the minority class leading to a total number of 10 datasets for our experiments (see Table  1 ). Then we separate the training and test sets (70% -30%) and, as we assume to have a low amount of data available for calibration, from the training set, we put aside a calibration set designed to have around 10 positive samples (so approximately 100 -1000 samples in total for the calibration set).
In Table  1 , we provide information about the datasets that have been used and their properties. For our experiments, we used a balanced bagging supervised classification algorithm based on a Random Forest (RF) estimator from the imblearn Python package  [28] , the EasyEnsembleClassifier  [25]. For each dataset, we made 20 repetitions where we trained an EasyEnsembleClassifier (with 120 RF base estimators) model with random split for train, calibration, and test sets. To evaluate only calibration and prevent us from prior shift, we split using stratifying. Once we have a trained model for a given dataset, we can apply traditional and new proposed calibration methods 1  and evaluate them through our different calibration metrics. Since we have related paired samples, we can apply Friedman statistic test  [36]  to evaluate if one method is significantly different from the others.
To do this, for each calibration metric, we formed blocks of errors for every repetition and dataset grouped by the different methods. Giving a p-value < 0. 05, we can reject the null  x ← random sample from D - 15: D ← D\{x} hypothesis that the different methods are similar. Furthermore, we use a posthoc analysis method such as Nemenyi  [37] [38] [39] , which allows us to identify the methods that differ significantly from others. However, it is important to note that the 20 repetitions for a given dataset are not perfectly independent since, for each repetition, we draw random training, calibration, and test sets from a unique original dataset (see line 5 in Algorithm 2). Therefore, we differ from the independence hypothesis required in the Friedman-Nemenyi test so that the results must be taken with caution and should be confirmed on additional datasets.
For each calibration metric and dataset, we also provided Wilcoxon tests between the best calibration method (on average) and the others. The experimental design is described in Algorithm 2. ▷ see  [25, 29]  10: Fit h on D tr 11: for f ∈ M do 12: Fit f on h(D cl ) 13: Compute calibration errors for each measure on D ts and save results.
Experimental results Some results are presented in Table  2 , Table  3  and Table 4  (with a polynomial degree of 10 for the polynomial calibration methods) for different metrics. Subject to the interpretation we wanted to have, looking at an average of ECE @k until 20% of the test data makes sense in the case of taking sensitive decisions on the top k predictions. We can observe that the new proposed framework and calibration methods deliver, in general, better results than the traditional ones, on the investigated datasets. The polynomial non-decreasing method (with a degree of 10) seems to be one of the best-investigated methods. On a dataset level, even though it is not always significantly better than Isotonic in particular, if we merge errors for each dataset, meaning that we have a set of 20 (number of repetitions) × 10 (number of datasets) = 200 values for each method, we can perform a rank test with Friedman and Nemenyi analysis at a global level.
This highlights that some of our methods unveil significantly better results than the traditional ones (see Figure  7 ). The new calibration framework for imbalanced classification, which consists in re-adjusting the posterior probability with eq. (  5 ) before applying calibration mapping, delivers good results as well. However, before rushing to conclusions, alongside graphical insights provided in Appendix B, we provide calibration results for ECE @10% and the classic ECE in Table  3  and Table  4. Table  2 : Avg-ECE @20% in percentage along datasets. For each dataset, we represent the average of Avg-ECE @20% along the 20 repetitions -the lower, the better. The zscore is performed to compare calibration methods only. We also performed a Wilcoxon test between the best method on average and the one into consideration and underlined errors that are not significantly lower than the best (p-value greater than 0.
05). Fig. 7 : Avg-ECE @20% -Friedman test and Nemenyi post-hoc analysis -Critical difference diagram of mean score ranks (ranging from 1 to 6). Grouping all datasets and repetitions, we perform a Frideman test between the calibration methods: p-value ≪ 0. 05. Thus, we provide a Nemenyi posthoc analysis which is represented graphically. It shows the mean score rank (the lower, the better) of each method and connects those which are not significantly different. The critical difference (CD) is also shown above the graph.
Dataset Given the results in Table  3  and Table  3  we can conjecture that our new proposed framework and Polynomial calibration method outperform traditional ones. Moreover, the new proposed framework can be applied in multi-class calibration to enhance existing parametric calibration methods such as Temperature scaling, Vector scaling and Matrix scaling  [2, 5]. Indeed, eq. (  5 ) can be generalized to multi-class classification  [27]. Alongside results shown in Appendix B, it reveals that when we look at the first percentage of the top k predictions (ECE @k% for k small), even the uncalibrated appears Table  3 : ECE @10% in percentage along datasets. For each dataset, we represent the average of ECE @10% along the 20 repetitions -the lower, the better. The z-score is performed to compare calibration methods only.
We also performed a Wilcoxon test between the best method on average and the one into consideration and underlined errors that are not significantly lower than the best (p-value greater than 0. 05). to be well calibrated. This means that the uncalibrated model is still well-calibrated in the very high confidence score intervals and for a small number of predictions. Nevertheless, by tuning the ECE to restrict it for the top k predictions, we could lose information and be biased when trying to measure calibration for a very small value of k. Indeed, to estimate the calibration error for a given bin we need a sufficient amount of predictions and if k is very small we may be subject to high per-bin variance. Moreover, unlike ECE @k , the classic ECE (3) measures calibration for the entire confidence score interval but it is not really well suited for highly imbalanced datasets because of its weighting scheme.
Dataset When trying to measure calibration on imbalanced datasets along the whole confidence score axis, we can notice quite different results for ECE, Table  4 , and ICE smooth , Table  5. Even though ECE @k% and ECE use the same weighting of bins approach, ECE @k% should focus more on one might want to address attention (i. e. calibration among high score predictions). Since Avg-ECE @20% seems to be more correlated with ECE than ICE smooth , we could debate on the consistency of the ICE metrics and whether we should rely on it or not compared to ECE @k% in an imbalanced set-up. ICE might measure something too far from the concept of calibration.
Declarations Conflict of interest statement. The authors declare that they have no conflict of interest. provided a 90% confidence interval related to a Student test. ECE @k , with a step of 10 for k, until 20% of total test data is provided. We do not show the graphical illustration for the Breast Cancer dataset because we have only 120 test samples. Thus, ECE @k would be plotted until k = ⌊ 120×0. 2 10 ⌋ × 10 = 20 with a step of 10, so there is not enough data to make a meaningful plot.
Algorithm 2 design 1 : 6 : 7 : 8 : 21678 Experimental D ← set of binary imbalanced datasets ▷ see Algorithm 1 2: M ← calibration method set 3: for D ∈ D do 4: for i = 1,. , 20 do 5: D source , D ts ← random 0. 7;0. 3 splitting of D nb pos calib ← 12 size calib ← nb pos calib |D + source | D tr , D cl ← random (1-size calib);(size calib) splitting of D source 9:h ← Pipeline StandardScaler, SelectKBest, EasyEnsembleClassifier (RandomForestClassifier).
This idea is also in agreement with another recent study investigating the effects of tTIS on motor sequence learning  52. Notably though, a limitation of the present dataset is the very short duration of stimulation and imaging for each experimental condition, that may explain some inconsistencies in the results. Hence, following this first proof-of-concept study showing robust behavioural effects and related neural changes, future studies including longer fMRI and stimulation sessions are required to further confirm these results. Finally, within the present study the computational modelling was performed on a realistic, detailed head model (i. e. , the MIDA model  59  , see Methods). One limitation of this approach is that the electric field simulations do not take individual structural information into account.
Imaging quality control A threshold of 0. 5 was chosen to discard subjects showing more than 40% of voxels with framewise displacement FD higher than this threshold. In the current study cohort, no subject exceeded the limit value, thus the whole dataset could be used. Furthermore, successful cleaning of the data was ensured by visual checking the preprocessing results. In particular, good registration between anatomical and functional images and normalization to standard space were checked. Signal to noise ratio analysis showed significantly higher tSNR values underneath the stimulating electrodes (F(1,1122)=249. 25, p<0. 001; Figure  S5 ). Moreover, an additional analysis showed that this effect was not influenced by the tTISTYPE (SphereLOCATION x tTISTYPE: F(2,1118)=0. 0169, p=0. 98). This result suggests that the stimulation did not introduce additional noise to the MR images.
Acknowledgements P. V. was a PhD student supported by the  Fund for Research training in Industry and Agriculture (FRIA/FNRS ;  FC29690 ), and grants by the  Platform for Education and Talent (Gustave Boël -Sofina Fellowships)  and  Wallonie-Bruxelles International. J. D. was supported by grants from the  Belgian FNRS  and the  Fondation Médicale Reine Elisabeth (FMRE). G. D. was supported by the  Belgian FNRS. The research was partially funded by the  Novartis Research Foundation-FreeNovation (Basel , CH) to MJW and EN, the  Bertarelli Foundation  (Catalyst ' Deep-MCI -T ', Gstaad, CH) to FCH, the  SNSF Lead Agency  ( NiBS-iCog 320030L_197899 ) to FCH and the  Defitech Foundation (Morges , CH) to FCH. We acknowledge access to and expertise of the Neuromodulation and the Neuroimaging facilities of the  Human Neuroscience Platform of the Campus Biotech Geneva  and of the  MRI Platform of the HVS (Sion).
Dielectric properties were assigned based on the IT'IS Tissue Properties Database v4. 0 122. Rectilinear discretization was performed, and grid convergence as well as solver convergence analyses were used to ensure negligible numerical uncertainty, resulting in a grid that included more than 54M voxels. Dirichlet voltage boundary conditions, and then current normalization were applied. The electrode-head interface contact was treated as ideal. tTIS exposure was quantified according to the maximum modulation envelope magnitude formula from Grossman et al. , (2017) 41. Then, a sweep over 960 permutations of the four electrode positions was performed, considering symmetric and asymmetric montages with parallel (sagittal and coronal) or crossing current paths, while quantifying bilateral striatum (putamen [BNA regions 225, 226, 229, 230], caudate [BNA regions 219, 220, 227, 228] and nucleus accumbens [BNA regions 223, 224]) exposure performance according to three metrics: a) target exposure strength, b) focality ratio (the ratio of target tissue volume above threshold compared to the whole-brain tissue volume above threshold, a measure of stimulation selectivity),.
Acknowledgements. The authors wish to thank  Météo-France  and the  SCHAPI , for making available the climate and hydrological data used in this study. CNR and  INRAE  are thanked for co-funding the PhD grant of the first author. Paul Astagneau  and  Laurent Strohmenger  are also thanked for their advice on the manuscript. The authors thank  Isabella Athanassiou  for copyediting the manuscript. https://doi. org/10. 5194/egusphere-2023-569 Preprint. Discussion started: 13 April 2023 c Author(s) 2023. CC BY 4. 0 License.
H Group was characterized by an herbaceous cover above 25% and by the absence of woody plants. W Group was characterized by the presence of woody plants and by an herbaceous cover below 25%. HW Group was characterized by an herbaceous cover above 25% and by the presence of woody plants. Finally, the last group was characterized by an herbaceous cover lower than 25% and by the absence of woody stratum. However, this group represented less than 1% of the vegetated linear feature, so was not considered in this analysis. Next, all green linear features were georeferenced using QGIS software. Human traffic routes include roads (paved traffic routes) and tracks (unpaved traffic routes). To map roads and tracks in our two sites, we used vectors from IGN cartographic database (ROUTE 500® edition 2020) and from OpenStreetMap database (OSM).
Acknowledgements We would like to thank all the people involved in the fieldwork among sites and over years and in particular  Frédéric Le Capitaine ,  Benoît Laboup ,  Arnaud Gaujard ,  Julie Grezeleau ,  Romain Mongeot ,  Estella Vignon ,  Anne-Charlotte Pommier-Petit ,  Soumaya Belghali  and  Lisa Gili. We would also like to thank our partners for their support in this study: the  Federation of Hunters of Haute-Garonne (FDC 31)  and the  Federation of Hunters of Tarn-et-Garonne  (FDC82). We thank  Aude Géraud  for her logistic support during the project. We also would like to thank  Françoise Ponce ,  Jean-Bernard Puchala  and  Luc Fruitet  for their help in setting up this project. We are grateful to  Michel Ruas  for the donation of 2 farmed red-legged partridges which allowed us to improve our skills in equipping birds with GPS tags.
To evaluate the potential duration of transmission for EG. 5. 1. 1 M49L-mutants, we calculated minimum clade-durations for all M49L-clades by calculating the difference in time between the earliest and latest sampling dates among sequences of a given clade. We found that cladeduration could be up to 58 days for M49L clades belonging to the EG. 5. 1. 1 lineage. This first assessment however, is based on a small number of clades (3) and on a limited sampling of the recent circulating diversity of SARS-CoV-2. For computational tractability reasons we focused our phylogenetic analysis on sequences from a single lineage (EG. 5. 1. 1), specific countries (Japan, North America), and subsampled our dataset. We believe that it is of the utmost importance to pursue these investigations and to continue to monitor the ability of M49L-carrying lineages to establish sustained transmission and to assess the risk of fixation of the mutation.
Acknowledgments We thank  Magali Gilles  ( UVE; Marseille ),  Gregory Moureau  ( UVE; Marseille ) and  Camille Placidi  ( UVE; Marseille ) for their valuable technical contribution. We thank  Edyth Parker  for her recommendations on SARS-CoV-2 phylogenetic inference. We thank  Pr. Drosten  for providing.
the SARSCoV-2 strain through the  European Research infrastructure EVA GLOBAL. This work was by  European Virus Archive Global  ( EVA GLOBAL ) funded by the  European Union 's  Horizon 2020 research and innovation program  under grant agreement No. 871029  and by  DNDi  under support by the  Wellcome Trust  Grant ref:  222489/Z/21/Z  through the  COVID-19 Therapeutics Accelerator. Part of the work was done on the  Aix Marseille University  antivirals platform "AD2P".
The content of this layer appears irregular within each core: it can contain 1 or several rows (Fig. 1b, f ) and is sometimes even absent. Strong densities are occasionally found to be connected to the top of the stakes (Fig. 1d ). Consistent with previous data and the gentle sample preparation procedure we used, these in vitro cores also contain two large blob-like structures on both sides: the lateral bodies. Our dataset confirms that the lateral bodies are amorphous and reveal some connection sites linking them to the spikes of the palisade layer (Fig. 1b , Movie S2). AlphaFold2 predictions of major core proteins AF2  [10]  was used to predict the 3D structure of five viral proteins known to be abundant in fractions of cores prepared in vitro  [4]. These are the gene products of A3, A4, A10, and L4 and the lateral body protein F17  [5].
2 ). To further confirm the visual similarity between the in vitro and in situ cores, nearestneighbor analysis was performed. The average distance between neighboring stakes is 8 nm for both datasets and the analysis of the normalized nearest neighbor position shows a similar trend (Fig. 4b, c ). This analysis confirms the irregular arrangements of the lattice in situ, since there is no visible positional preference of the nearest neighbors. The number of neighboring stakes within a given distance per trimer shows that for both data sets only around 5% of the stakes form a hexagonal arrangement (Fig. 4c ). The STA maps of pairs of trimers from in vitro cores show that even the neighboring trimers are not consistently oriented towards each other (Fig. 4d ). The average number of stakes per core is 2690 for the in vitro data.
Preprocessing and tomogram reconstruction The defocus for each tilt series was estimated with CTFFind4  [21]. The tilt series were doseweighted using the MATLAB script from  [22]. Suboptimal projections or whole tilt series were removed from dataset, resulting in 34 tilt series. They were aligned using fiducial-based alignment within eTomo. The tomograms were reconstructed from 8x binned tilt-series using WBP with SIRT-like filter option. These tomograms were used only for picking of cores within Napari (see below). The unbinned tomograms were reconstructed using novaCTF  [23]  with phaseflip correction, astigmatism correction, and a 15nm slab size. These tomograms were binned 2x, 4x, and 8x using Fourier3D  [24]. These 3D-CTF-corrected tomograms were used for subtomogram averaging (STA). For the complete parameter setup of individual steps see Table  S2.
Subtomogram averaging (in situ) The STA on in situ data was performed in the same way as for the in vitro dataset with the exception of the initial alignment, which was skipped. Instead, after segmenting the cores, the STA was directly performed with AF2 model of A10 trimer. After the localization, the new orientations were assigned, followed by STA using STOPGAP. The trimer was resolved to the resolution of 13. 4Å (Fig  4 ). The relatively low number of particles (6325) prevented any attempts for classification. Table  S2  contains all the parameters for the in situ STA.