dataesr/scientific-paragraphs-categorization huggingface dataset:
Replicating the original study, we also performed a permutation test to assess the predictive performance. In short, we randomly shuffled the BSID-III scores 1000 times to generate null distributions for random predictors for both the Pearson's correlation coefficient and MAE. Pvalues of observing the reported r (or MAE) by chance is then calculated as the ratio of number of permutation tests with r higher than the observed r value over the number of permutation tests (reverse for the MAE). Additionally, to test the impact of random small differences in the dataset on the robustness of the predictive results, we randomised the inputs and outputs 100 times and repeated the same evaluation to derive the mean and standard deviation of predictive results across the independent runs to determine how sensitive the results are to small changes in the inputs upon shuffling.
Hyperparameter tuning & nested validation (Cohort D) The larger size of the dHCP available data compared to the original work allows us to perform a more thorough validation of the predictive pipeline (Fig. 2 ). To do this, we randomly selected a subset n times (n = 10) of 46 infants from the Cohort C keeping them as an independent testing set (to recreate the validation from the original study). The remaining data, i. e. 80 subjects, was used within the inner validation LOOCV loop (79 subjects to train and one as validation) which allowed us to tune the hyperparameters of the SVR. The best model was then retrained on the 80 subjects and evaluated on the independent testing split. The mean R2 and MEA results across the 10 repetitions are reported as the final predictive results. Again, the prediction of GA at birth and the binarized outcome scores were used as the sanity check and performed the same evaluation in the form of a categorical predictive task.
Therefore, all the following reported results are based on the inputs corrected for the PMA at scan. Interestingly, the random parcellations led to consistently higher predictive results than the atlas parcellation with a tendency of the predictive power to increase with number of regions. This is possibly due to larger number of parcels providing more granular information than when collapsed too much within the atlas parcels. Another observed tendency was the general trend of decreasing predictive power across the cohorts as the dataset gets progressively limited in variability of inputs and sample sizes. Interestingly, in the case of Cohort D, which is the closest replication of the original study, we observed a substantial drop in predictive power of prematurity status. Comparing these results to those in the Cohort C suggests this is largely due to insufficient sample size leading to negative results rather than the limited variability within the data given the similar distribution of input and output features.
We also performed a similar prediction aiming to categorise infants into three classes by additionally subdividing the preterm group of Cohort A into extreme-very preterm (EVP, GA at birth <32w, N = 8) and moderate-late preterm (MLP, GA at birth ≥32w, N = 35) groups. In this setting, only the atlas parcellation led to higher than random prediction (Supp. Table  4 ). Results are in line with previous findings demonstrating the differential effect of the degree of prematurity on brain measures and suggest that the categorization within the preterm group is possible. Nevertheless, it is important to note that practical issues, such as size of the dataset or class imbalance, are likely to impact the reliability of these observations, at least in our limited sample. When attempting to predict continuous GA at birth rather than categorised prematurity status, the predictive models did not outperform random levels (Supp.
BSID-III score prediction Our results of the predictive models for BSID-III outcome suggested limited ability to distinguish Atypical vs Typical categories for cognitive, language, and motor composite scores (Fig. 5b ). All models generally performed poorly, with the performance depending on the combination of the cohort and parcellation strategy. Once again, the random parcellation seemed to lead to better results supporting the idea there might be more granular information than that contained within 52 atlas parcels. However, the results were too heterogenous and close to random levels to make conclusions on optimal number of parcels with confidence. The erratic behaviour observed for Cohort D (especially for language and motor scores) might be due to the lower stability of the model to the small variations within the data caused by the randomly sampled small dataset, i.
Discussion In this study, we were unable to conceptually replicate the findings of  Ouyang et al. (2020)  that cortical microstructure at birth as assessed by DTI-based FA can reliably predict later neurodevelopmental outcome as evaluated with BSID-III in infants born preterm and full-term. Nevertheless, the performance on the prematurity task allowed us to draw limited conclusions: i) a sufficiently large and heterogeneous dataset appeared to be an indispensable factor for successful training and reliable validation of ML algorithms; ii) age corrections of the cortical FA features appeared to be required, and iii) random parcellations with higher number of parcels seemed beneficial compared with atlas based parcellations. Differences between the original study and our observations are discussed below in the light of deviations between methodologies and possible limitations.
g. through regional value interpolation at TEA, Fig. 3c ) could be done in the future to keep the inherent but potentially informative differences between the cortical parcels. For the same reason, it might also be worthwhile scaling all input FA features across regions simultaneously, as here (like in the original study), each input FA feature was scaled to lie in the range (0,1) over the group for each parcel independently. Another important methodological aspect for prediction studies relies on the implemented pipeline. Results of ML studies are by nature, highly dependent on the dataset and its learning procedure, and on data splitting strategies. Approaches can be highly variable and thus results often generalize poorly to other datasets, especially in the context of small datasets  (Woods, 2018; Varoquaux, 2018; Poldrack et al.
, 2020). In our case, even the largest Cohort A, which is six times larger than the dataset in the original study, would typically be considered small in a ML context and might also suffer from these generalization limitations. We attempted to remedy this by a thorough assessment of predictive performance which is a crucial step to dissociate whether the reported results are reliable or whether they might stem from performance fluctuations. Although the leave-one-out cross-validation strategy is popular for small datasets since it maximizes the training set size by keeping only a single subject for the testing, its main limitation is reduced estimation of generalization performance due to the depleted test set  (Bouthillier et al. , 2021). We thus aimed to leverage our larger cohort which allowed us to assess how, holding the algorithm constant, the performance changes under small perturbations to the data by performing randomized repetitions of the k-fold validation.
However, the high quality of the dHCP dataset for anatomical and diffusion MRI has been highlighted by several studies from different groups. We are also confident about the reliability of our measures and analyses having observed successful prediction of prematurity status based on cortical FA (corrected for PMA at scan), even in reduced cohorts. Besides, development of cortical microstructure is defined by multiple ongoing processes such as the growth of dendritic arborisation, synaptogenesis, and myelination of intracortical white matter, combined with the large-scale complex process of cortical folding. As in the original study, we focused on the widely used DTI-derived FA measure. In the preterm period, cortical FA is initially high due to the early radial orientation of apical dendrites and radial glia, and then decreases: which has been suggested to reflect increased neurite growth in all orientations as the cortex matures  (McKinstry et al.
In contrast to the original study which used the BSID-III scores assessed around 2 years of corrected age, the neurodevelopmental outcome assessment available in the dHCP database was performed at around 18 months. The overall neurodevelopmental stage in terms of language, cognitive and motor acquisitions between the two ages is very similar and is unlikely to change the significance or conclusions derived from our results. Additionally, assessment at 18 months corresponds to a strategic age for identifying neurodevelopmental impairment in preterm infants, and thus examining whether cortical microstructure at birth relates to outcomes at this age is of interest even in low-risk preterm populations without obvious brain anomalies who can still develop subtle neuromotor disorders (e. g. developmental coordination disorder)  (Groeschel et al.
, 2019; Spittle and Orton, 2014; Edwards et al. , 2011; Zwicker et al. , 2012). Although in our study, prematurely born infants were over-represented in the atypical category of the BSID-III motor score compared with full-terms, no major development delays or specific disabilities were neither expected nor observed within the specific low-risk cohort analyzed in our replication study. Therefore, an important question raised in the original study of insufficient heterogeneity within the input and outputs remained pertinent in our work. As an extension, we performed the training and evaluation of the outcome predictions on different cohorts with expanded variability for the targets (Cohort B) as well as both targets and inputs (Cohort A), but without success in terms of prediction results. The limited variance within the input features combined with the relatively narrow distribution of the BSID-III scores across the dataset may have still been insufficient to allow the ML models to establish relationships with outcomes even if such relationships do exist.
Conclusions Machine learning has become a major focus of research as a promising tool for early behavioural outcome prediction. However, very few clinical questions, including the neurodevelopmental assessments, represent well posed discrimination tasks that can be naturally framed as ML problems  (Baker and Kandasamy, 2022). Careful consideration of potential confounders that might drive predictions instead of the inputs of interest is required before ML can become useful in the clinical practice. This is further compounded by a lack of standardized methodologies for model implementation with some validation methods in current ML applications leading to highly variable and potentially inflated ML performance estimation, in particular in small datasets  (Varoquaux, 2018). As in many other areas, reproducibility failures in ML-based science appears prevalent without any systemic solutions  (Kapoor and Narayanan, 2022).
K. Siampos, A. Stergiou and P. West for useful discussions and feedback. Nehal Mittal acknowledges the of the Ecole Polytechnique for hospitality during his M1 internship in 2021 and DIM Quantip for funding his PhD fellowship at the LKB. The work of D. Rivera-Betancour was funded by Becas Chile (ANID) Scholarship No. 72200301. The work of M. Vilatte was supported by the Hellenic Foundation for Research and Innovation (H. F. R. I. ) under the First Call for H. F. R. I. Research Projects to support Faculty members and Researchers and the procurement of high-cost research equipment grant (MIS 1524, Project Number: 96048).
Objective: Little is known about the longitudinal relationship between carpal tunnel syndrome (CTS) and mental health. Therefore, this retrospective cohort study investigated the association between CTS and the five-year incidence of anxiety disorder and depression in adults from Germany. Methods: Data from the Disease Analyzer database (IQVIA) were used for the present study. Patients aged ≥18 years diagnosed for the first time with CTS in one of 1284 general practices in Germany in 2005-2020 were included (index date). Individuals without CTS were matched to those with CTS using a propensity score based on age, sex, the mean number of consultations per year during the follow-up, and the index year. In people without CTS, the index date was a randomly selected consultation in 2005-2020. Results: There were 75,135 patients with and 75,135 patients without CTS included in the study (mean [SD] age 57.
Database The Disease Analyzer database (IQVIA) was used for the present study. The methodology of this database has been previously detailed in the literature  [22]. Briefly, the Disease Analyzer database is a proprietary database that contains demographic data, diagnoses, and prescriptions anonymously collected from general and specialized practices in Germany. Data are transferred from the computers of the practices to IQVIA every month. Diagnoses are coded using the International Classification of Diseases, 10th revision (ICD-10), while prescriptions are coded using the Anatomical Classification of Pharmaceutical Products of the European Pharmaceutical Market Research Association (EphMRA). The quality of the data is assessed based on several criteria, such as completeness of information and linkage between diagnoses and prescriptions.
Interpretation of the findings Before going further, one should bear in mind that not all statistically significant results are clinically pertinent. The effect size of the relationship between CTS and incident anxiety disorder was small, and this finding should be interpreted with caution. That being said, the study is in line with the previous research focusing on the association between CTS and mental health  [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]. For example, a study, including 91 cases with CTS and 41 controls without CTS in the United Kingdom, showed that depression but not anxiety scores were significantly higher in cases than in controls, these scores being assessed with the Hospital Anxiety Data are percentages unless otherwise specified. Fig. 2. Five-year cumulative incidence of anxiety disorder in patients with and without carpal tunnel syndrome (Kaplan-Meier curves).
Conclusions To conclude, in this retrospective cohort study of >150,000 adults from around 1300 general practices in Germany, there was a positive and significant association between CTS and the five-year incidence of anxiety disorder and depression. These results suggest that anxiety and depression symptoms should be regularly assessed after the diagnosis of CTS. Finally, more research is warranted to corroborate or invalidate these findings in other countries and settings with different cultural backgrounds. Supplementary data to this article can be found online at https://doi. org/10. 1016/j. jpsychores. 2023. 111469.
ACKNOWLEDGMENTS We thank  Benoît Douçot  and  Andrea De Luca  for useful discussions. M. F. thanks  Sounak Biswas  for clarifying some aspects of jackknife resampling. T. S. was supported by a  James Buckee scholarship. D. B. was supported in part by  CNRS , by the  ENS , and by the  ANR  project " ESQuisses ," Contract No. ANR-20-CE47-0014-01. A. N. was supported by  CNRS  and the  École normale supérieure.
Introduction The contribution of single particle cryo electron microscopy (cryo-EM) to the structural biology field has been steadily increasing over the past decade, with a strong growth in the last few years as illustrated by the number of derived structures and atomic models that are deposited in the protein data bank (PDB) and electron microscopy data bank (EMDB), and specifically of large complexes which are more easily amendable to cryo-EM analysis than to X-ray crystallography (Fig. 1A, B ). Of note, medium and high-resolution cryo-EM structures are all determined from data collected using electron microscopes operating under cryo conditions to preserve samples in a frozen-hydrated state. The proportion of high-resolution cryo-EM structures is increasing (Fig. 1C ) and hence allows detailed insights into structures, their secondary structure and the position of the individual side-chains.
This is important for a reliable derivation and validation of atomic models using various software (e. g. Phenix  (Liebschner et al. , 2019; Klaholz, 2019) ; Refmac  (Murshudov et al. , 2011) , Buster  (Smart et al. , 2012)  that allow detailed interpretation of the underlying molecular mechanisms, protein-protein or protein-RNA/DNA interactions, and inhibitor interactions within ligand binding pockets relevant for drug design. This impressive move of the cryo-EM field around the years 2013/2014 has become possible thanks to technological developments both on hardware and software sides, in particular since the introduction of direct electron detectors and CMOS (complementary metal-oxide semiconductor) cameras in ~ 2012/2013 (noticeable in the steep increase from 2013 onwards, Fig. 1C ), which, combined with advanced image processing such as 2D & 3D classification methods to sort out different conformations and compositions of the structures  (Klaholz et al.
) along with automated data collection, (iv) improving the imaging system at different levels, such as the electron source (cold field emission gun  [FEG] ) with smaller energy spread and low divergence which improves beam coherence and brilliance, phase plates (installed at the back focal plane together with the objective aperture), possible Cs (spherical aberration) correctors, energy filters, constant power lenses (higher beam coherence) and highsensitivity cameras. Energy filters and CMOS cameras are often considered to be accessory pieces of equipment, but the so-called Omega filter on Jeol instruments has long been an integral part of the microscope. Only recently, Thermofisher Scientific (TFS, previously FEI and originally Philips) developed a post-column energy filter, the so-called Selectris; before, the usage of an energy filter on microscopes such as Polara or Titan Krios required a post-column installation from a different company, e.
, 1983)  during grid loading on the microscope and data collection when the sample is mounted on the goniometer of the microscope. This implies a series of practical and technical implementations on the microscopes for preserving low temperatures (typically below -165 • C) throughout the process of sample transfer & image acquisition to avoid devitrification and ice contamination and also maintain drift rates low during data collection by using appropriate insulation materials and constant cryogenic supply with liquid nitrogen. Hence, while the contribution to data banks from structures solved using image data from either TFS or Jeol microscopes is rather unequal (Fig. 1D ), recent progress indicates that both companies provide similar tools (cold FEG, in/post-column energy filter etc. , Fig.
1E ). However, they have not been compared directly side-by-side yet with regards to performance, particularly considering that the integrated technologies are implemented rather differently. Testing cryo electron microscopes is usually done with standard samples such as cross grating or oriented, micro-crystalline gold to test the optical system and maximum nominal resolution, and standard protein crystals to test isotropic magnification and calibrate magnification (catalase, tobacco mosaic virus etc. ). In the vast majority of cases, simple test samples such as apo-ferritin or symmetric objects have been used  (Bartesaghi et al. , 2015; Nakane et al. , 2020; Yip et al. , 2020; Zhang et al. , 2020)  to check a microscope through a routine data collection run and estimate the resolution that can be reached in 3D reconstructions through image processing.
, 2019; Merk et al. , 2020; Fislage et al. , 2020; Wu et al. , 2020; Hamdi et al. , 2020; Adachi et al. , 2021; Efremov and Stroobants, 2021; Zhang et al. , 2021; Kato et al. , 2018; Maki-Yonekura et al. , 2021) , including also the 200 kV machines such as Glacios or Talos Arctica (TFS) and  CRYO ARM 200 (Jeol) , but no direct comparison has been done yet on the 300 kV high-end setups. In this work we analyse the performances of the two high-end cryo electron microscopes from TFS and Jeol, the Titan Krios and the CRYO ARM 300, both with the latest available options such as cold FEG emitter, post-or in-column energy filters, automated sample upload and automated data collection. As a test sample we used the human ribosome, a real case biological sample without internal symmetry of which we have determined the structure in the past  (Khatter et al.
Results & discussion Two cryo transmission electron microscopes were used for the present benchmarking, the Titan Krios G4 (TFS) and the CRYO ARM (Jeol), both equipped with energy filters (Selectris-X and Omega, respectively, see scheme in Fig. 1E ) and two high-end direct electron detector cameras operating in counting mode to measure individual electron events, the Falcon 4 and Gatan K3 for the TFS and Jeol microscopes respectively, where the Falcon 4 (TFS) is integrated in the system comprising a Selectris X energy filter. Both instruments were operated at an acceleration voltage of 300 kV, and both contained a cold FEG source, called E-CFEG (TFS) or CFEG (Jeol). An automatic liquid nitrogen filling system minimizes personal intervention and facilitates continuous data collection. Automated upload of the cryo-EM grids is done by a cryogenic sample manipulation robot called "autoloader" (TFS) or "automated cryo-transfer system" (Jeol), into which a small dewar with the pre-mounted specimen grids is introduced.
Human 80S ribosome sample grids were prepared according to our previously published procedures  (Khatter et al. , 2014; Khatter et al. , 2015; Myasnikov et al. , 2016; Natchiar et al. , 2017)  (see methods) and sent in a dry shipper for data collection at the TFS factory in Eindhoven and at the RIKEN Osaka cryo-EM facility proposed by Jeol (giving rise to data sets and 2, respectively, as named in the following). The microscopes were aligned for parallel beam conditions using coma free alignment procedures and data were collected in automated manner with either the EPU software (TFS) or the academic software SerialEM  (Mastronarde, 2003)  installed on the computer operating the Jeol instrument (see methods). Köhler illumination  (Köhler, 1893)  was used to remove interference fringes (called fringe-free imaging or zero fringe system by TFS or Jeol, respectively) to facilitate collecting several images per hole (multi-spot or "multi-shot").
Beam tilt compensation (Jeol) or aberration free image shift (AFIS, TFS; beam-image shifts combined with off-axis coma and astigmatism correction) were used to allow acquisitions on several neighbouring holes without stage movement (multi-hole). These acquisition modes  (Cheng et al. , 2018; Weis and Hagen, 2020)  help maintaining mechanical stability & low drift rates and increase data collection speed. Considering that the cameras used have rather different physical pixel sizes the magnification was chosen such that a similar pixel size on the specimen was obtained and fine enough to enable high-resolution structural analysis (0. 72 and 0. 82 Å / pixel for data sets 1 and 2, giving a Nyquist frequency of 1. 4 and 1. 6 Å, respectively). The specific devices and parameter settings used for data collection are summarized in Table  1.