dataesr/scientific-paragraphs-categorization huggingface dataset:
, 2019b) , with all 19 substitutions that generated some wild-type transcripts being listed in Table  2. By comparison to the commonly used minigene splicing assay, FLGSA preserves better the natural genomic sequence context of the studied variants  (Wu et al. , 2017; Zou et al. , 2016). The accuracy and reliability of the FLGSA-derived data can be inferred from the following three lines of evidence. First, 10 GT>GC substitutions that generated wild-type transcripts and 10 GT>GC substitutions that did not generate wild-type transcripts in transfected HEK293T cells were further analyzed in transfected HeLa cells using FLGSA, yielding entirely consistent findings in terms of whether or not wild-type transcripts were generated  (Lin et al. , 2019b). Second, HESX1 c. 357+2T>C and SPINK1 c. 194+2T>C were the only variants common to both the first and second datasets.
The functional effects of these two variants in vivo -HESX1 c. 357+2T>C generated no wild-type transcripts whereas SPINK1 c. 194+2T>C generated some wild-type transcripts (Supplementary Table  S1 )were faithfully replicated in FLGSA (Supplementary Table  S2 ). Third, a GT>GC variant that was not present in either dataset, HBB c. 315+2T>C, had been reported to be associated with a milder hematological phenotype and it was suggested that it might have a limited impact on splicing  (Frischknecht et al. , 2009). Using FLGSA performed in HEK293T cells, we found that it generated a low level of wild-type transcripts  (Lin et al. , 2019b). Importantly, the orthologous variant of HBB c. 315+2T>C in the rabbit Hbb gene has also been found to be capable of generating wild-type transcripts in two experimental model systems, namely in vitro transcription of Hbb RNA in a HeLa cell nuclear extract and transient expression of the full-length Hbb gene in HeLa cells  (Aebi et al.
, 1986; Aebi et al. , 1987). These notwithstanding, tissue-or cell-specific factors have on some occasions impacted splicing  (Jaganathan et al. , 2019; Pineda and Bradley, 2018) , an issue that was not extensively addressed in our previous study  (Lin et al. , 2019b). The bottom line here is that (i) the 30 genes used for FLGSA analysis were selected using a procedure that did not take into consideration the gene's function or expression, (ii) all 30 genes underwent normal splicing in the context of their reference mRNA sequences as specified in Supplementary Table  S2  and (iii) the generation (or not) of wild-type transcripts from the engineered GC allele was observed under the same experimental conditions as for the wild-type GT allele  (Lin et al. , 2019b). The third dataset was obtained courtesy of a perusal of the literature (Table  3 ).
Selection and interpretation of SpliceAI Delta scores for analysis We processed GT>GC variants using the default settings of SpliceAI as detailed in https://pypi. org/project/spliceai/. SpliceAI provides Delta scores (ranging from 0 to 1) for each variant, thereby providing a measure of their probability of altering splicing in terms of either splice donor gain, splice donor loss, splice acceptor gain, and splice acceptor loss. SpliceAI also provides Delta position that conveys information specifying the location where splicing differs from normal relative to the position of the associated variant. Since the GT>GC variants or substitutions under study invariably affected the +2 position of the canonical 5' splice site GT dinucleotides (in the context of the specified mRNA reference sequence), we focused our analysis exclusively on the Delta scores of donor loss although other scores may provide clues as to the nature of the resulting aberrantly spliced transcripts of splice-altering variants.
Encouraging findings from a quick survey of the three datasets of GT>GC variants As mentioned in the Introduction, none of the four most popular splicing prediction tools, SpliceSiteFinder-like, MaxEntScan, NNSPLICE and GeneSplicer, were found to be able to distinguish those GT>GC variants that generated wild-type transcripts from those that did not  (Lin et al. , 2019b). As described below, a quick survey of SpliceAI-predicted scores yielded encouraging results across all three datasets of GT>GC variants. First, in the context of dataset 1, the level of variant allele-derived wild-type transcripts associated with the seven disease-causing GT>GC variants was at most 15% of normal (Table  1 ). Although this low level increase in the generation of wild-type transcripts may make prediction a daunting task, it is interesting to see that the two lowest Delta scores of donor loss, 0.
, 2017). The score of 0. 63 was observed for the DMD c. 8027+2T>C variant, for which the detection of wild-type transcripts was based upon RT-PCR analysis of disease-affected muscle tissue from a hemizygous carrier with Becker muscular dystrophy  (Bartolo et al. , 1996). As for the second dataset (Supplementary Table  S2 ), the four lowest Delta scores of donor loss (i. e. , 0, 0. 03, 0. 05 and 0. 08) were all found in substitutions that generated wildtype transcripts; and 63% (n = 12) of the 19 substitutions that generated some wild-type transcripts had a Delta score of <0. 80 (Table  2 ). As for the third dataset, the lowest Delta score, 0. 53, was observed in association with the only "functional" BRCA1 IVS18+2T>C variant (Table  3 ).
Statistical comparison of experimentally obtained functional data with SpliceAI predictions for the 103 engineered GT>GC splice variants (dataset 2) Dataset 2 comprised 19 substitutions that generated wild-type transcripts and 84 substitutions that generated no wild-type transcripts. We thus performed two statistical tests, a Receiver operating characteristic (ROC) curve and a Matthews correlation coefficient (MCC), on the 103 substitutions assessed by SpliceAI (Supplementary Table  S2 ) with a view both to identifying an optimum threshold value and to assessing the correlation between the FLGSA assay results and the SpliceAI predictions. Based on an ROC analysis of 103 variants from dataset 2 (Supplementary Table  S2 ), an optimum threshold point of 0. 85 was provided -similar to the threshold of 0.
80, recommended by SpliceAI for high precision results. A contingency table was constructed (Supplementary Table  S3 ) to calculate values for the false positive rate, specificity, sensitivity, accuracy and the Matthews correlation coefficient. These are summarized in Table  4 , along with the AUC result obtained from the ROC analysis, the curve from which is shown in Fig. 1. As can be seen from Table  4 , the AUC of 0. 79 and the MCC score of 0. 41 are indicative of a good correlation between predicted and actual results. There is also a low false positive rate whilst still maintaining a high accuracy and sensitivity. These results show that for dataset 2, at a threshold of 0. 85, SpliceAI can accurately discriminate between those GT>GC substitutions which disrupt splicing and transcript production and those which do not disrupt splicing and produce transcript.
Considerable discrepancy between the predicted and experimentally obtained functional impact assessments of GT>GC 5' splice site variants Employing 0. 85 as the threshold Delta score (donor loss) to define the generation of wildtype transcripts, rather variable performance between SpliceAI-predicted and experimentally demonstrated functional effects of GT>GC variants were observed across the three datasets: 33-84% of the variants that generated wild-type transcripts and 67-89% of the variants that generated no wild-type transcripts were correctly predicted by SpliceAI (Table  5 ). The poorest performance (43% (3/7) and 33% (1/3)) was observed with datasets 1 and 3 variants that generated wild-type transcripts (Table  5 ). In the context of the seven dataset 1 variants that generated wild-type transcripts (a qualitative property), the relatively poor performance of 43% might be related to the fact that the functional impact of these GT>GC variants actually manifested as rather small quantitative changes, generating between 1-15% normal transcripts (Table  1 ).
This notwithstanding, it should be pointed out that the two disease-causing variants that generated 10-15% wild-type transcripts, CAV3 c. 114+2T>C  (Muller et al. , 2006)  and CD40LG c. 346+2T>C  (Seyama et al. , 1998) , had Delta scores of ≥0. 9 (Table  1 ); and in each of these two cases, RT-PCR analysis was performed using patient-derived and pathologically relevant tissue or cells. In short, it remains unclear why some of the disease-causing variants that generated comparable levels of wild-type transcripts were predicted to have low Delta scores (i. e. , DMD c. 8027+2T>C and SPINK1 c. 194+2T>C) whereas others were predicted to have high Delta scores (i. e. , CAV3 c. 114+2T>C and CD40LG c. 346+2T>C). In the context of dataset 3 substitutions that generated wild-type transcripts, the precise levels of wild-type transcripts generated by the two "intermediate" BRCA1 +2T>C substitutions (both had a Delta score of ≥0.
93; Table  3 ) were unknown. As for variants that did not generate wild-type transcripts, an excellent correlation rate, 89%, was observed with the 38 such disease-causing variants. By contrast, the performance in datasets 2 and 3 variants was much lower and almost identical (68% and 67%, respectively; Table  5 ). A fundamental difference between dataset 1 variants and the latter two dataset substitutions is that all of the former were previously published whilst almost all of the latter were prospectively generated. Thus, it is tempting to speculate that for most of the 38 disease-causing variants that did not generate wild-type transcripts, their functional effects might have been 'seen' by SpliceAI during training, thereby leading to a higher correlation rate. In an attempt to further understand the poor performance of dataset 2 and 3 substitutions that did not generate wild-type transcripts, we opted to use the corresponding +2T>A and +2T>G substitutions as controls.
The underlying premise was that, based upon current knowledge, +2T>A and +2T>G variants should completely disrupt normal splicing and consequently have high Delta scores in virtually all cases (see also section 3. 6). Here it is worth mentioning that we previously employed FLGSA to analyze the functional impact of 15 +2T>A substitutions and 18 +2T>G substitutions, none of which generated any wild-type transcripts  (Lin et al. , 2019b). We processed all corresponding +2T>A and +2T>G variants by means of SpliceAI in the same way as for the +2T>C variants (during October 2019), the resulting Delta scores for donor loss being provided in Tables 1-3 and Supplementary Tables  S1  and S2. As shown in Supplementary Table  S1 , all +2T>A and +2T>G variants corresponding to the 45 disease-causing +2T>C variants had very high Delta scores, ranging from 0.
92 to 1. By contrast, 91% (n = 94) of the +2T>A and +2T>G variants corresponding to the 103 dataset 2 +2T>C substitutions had a Delta score of ≥0. 85 (Supplementary Table  S2 ). In other words, nine of the 103 +2T sites were predicted to have a Delta score of <0. 85 when substituted by either A or G; and in these sites, the Delta scores are often identical for all three possible substitutions (Table  6 ). One possible reason for lower than expected Delta scores is provided in  (Jaganathan et al. , 2019) ; exons which undergo a substantial degree of alternative splicing, defined as being between 10% and 90% exon inclusion averaged across samples, tend towards intermediate scores (stated as between 0. 35 and 0. 8). We therefore explored this possibility using the two sites for which all possible substitutions had the lowest Delta scores (i.
, 2018). However, of the three +2T>A substitutions that were experimentally shown to generate some wild-type transcripts, two were predicted to have a Delta score of >0. 85, namely 0. 93 for HESX1 IVS2+2T>A (Supplementary Table  S2 ) and 0. 98 for BRCA1 IVS18+2T>A (Table  3 ). The other one, BRCA1 IVS16+2T>A, was predicted to have a Delta score of 0. 74; but an identical score was also predicted for BRCA1 IVS16+2T>C and IVS16+2T>G, both of which were classified as "non-functional" (Table  3 ). In short, SpliceAI appeared not to work as well for the +2T>A variants that generated wildtype transcripts as for the +2T>C variants that generated wild-type transcripts.
CONCLUSIONS AND PERSPECTIVES In the present study, we attempted to correlate SpliceAI-predicted and experimentally obtained functional effects of GT>GC variants in the context of three independent and complementary datasets. Employing data from dataset 2 substitutions, we were able to propose a Delta score of donor loss, 0. 85, as defining the threshold of whether or not wildtype transcripts would be generated by GT>GC variants; whereas a score of ≥0. 85 defines the complete absence of wild-type transcripts, a score of <0. 85 defines the generation of at least some wild-type transcripts. Subsequent use of this threshold score to correlate SpliceAI-predicted and experimentally obtained functional effects of the GT>GC variants revealed that SpliceAI performed better than the popular prediction tools such as SpliceSiteFinder-like, MaxEntScan, NNSPLICE and GeneSplicer.
However, a considerable discrepancy still existed between SpliceAI-predicted and experimentally obtained functional assessments in relation to GT>GC (as well as GT>GA) variants. Indeed, this discrepancy serves to illuminate the challenges ahead in accurately identifying all splice-altering variants. A key issue here is that the impact of GT>GC (as well as GT>GA) variants that generated wild-type transcripts represents a quantitative change that can vary from barely detectable to almost full expression of wild-type transcripts, with wild-type transcripts often co-existing with aberrantly spliced transcripts. This is also the case for most of the splice-altering variants occurring outside the essential splice site dinucleotides, whose effects "are not fully penetrant and a mixture of both normal and aberrant splice isoforms are produced"  (Jaganathan et al.
, 2019). Moreover, there is also the issue of alternative splicing related to tissue-or cell-specific factors. While it is clear that we are still very far acquiring a full understanding of the 'splicing code'  (Bao et al. , 2019) , we are of the opinion that any improvement in the prioritization of splicing variants will necessitate the refinement of in silico prediction tools by reference to in vitro functional assessment courtesy of the results obtained from well-validated assays such as FLGSA. Figure  1. A receiver operating characteristic (ROC) curve for the SpliceAI predictions generated from dataset 2 (Supplementary Table  S2 ), with dotted diagonal line indicating a random prediction (0. 5 AUC) and the solid line showing SpliceAI prediction performance (0. 79 AUC). The intersection between the two represents the optimum threshold.
Variants generating no wild-type transcripts Splice AI Delta score (donor loss) of 0. 85 was used as the threshold value for defining the generation of wild-type transcripts or not. Dataset 1 (45 disease-causing variants) 89% (34/38) Dataset 2 (103 variants analyzed by FLGSA) 68% (57/84) Dataset 3 (12 BRCA1 variants analyzed by saturation genome editing) 67% (6/9) *.
In particular, we explored whether transmodal association cortices functionally shifted towards sensory areas in ASD, a pattern that would make sensory input harder to ignore, and that would compromise higher-order cognition by preventing the segregation of internally-driven cognitive processes. Our analysis of a large multi-centric dataset of individuals with ASD and typically-developing controls  23, 24  revealed perturbed functional gradients in autism, showing reduced functional distance between transmodal and unimodal regions. Notably, while ASD displayed an initial acceleration in SFC from sensory to early heteromodal systems, transitions into transmodal regions were aberrant, and failed to converge on the DMN core. Gradient and SFC findings in ASD were also contextualized with graph theoretical metrics, specifically connectivity distance and rich-club features  9  , offering a complementary viewpoint on cortical hierarchy i.
e. , the dissociation of a transmodal core that aggregates long-distance connections from peripheral networks with mainly short-range connectivity. Atypical gradients and stepwise connectivity in ASD related mainly to a selective disruption in longrange connectivity, co-occurring with a deficit to fully activate the rich-club. Finally, supervised pattern learning successfully leveraged hierarchy features to predict symptom severity in social cognition and repetitive behavioral symptom domains, as indexed by the ADOS (Autism Diagnostic Observation Schedule  25  ) in individuals with ASD. Effects were reproducible across included sites, in an independent validation dataset, and with respect to several MRI-specific processing choices. Taken together, our study shows that ASD is characterized by perturbations in macroscale cortical hierarchy which provides a parsimonious account of its paradoxical combination of low-level and higher-order symptoms.
Results Data samples. We studied two independent subsamples from the openly-shared Autism Brain Imaging Data Exchange initiative (ABIDE-I and II; http:/fcon_1000. projects. nitrc. org/indi/abide)  23, 24. Our discovery dataset originally included 143 individuals with ASD and 144 age-matched healthy controls, aggregated across those sites from ABIDE-I that included both children and adults (i. e. , PITT, NYU, USM). MRI quality and preprocessing were visually and quantitatively evaluated. Excluding cases with low-quality structural MRI or inaccurate cortical surface extraction (visually checked and manually corrected by SLV and BCB) or high head motion on rs-fMRI (mean framewise displacement >0. 3 mm, 2 SD from the mean across all subjects) resulted in a final sample of 103 ASD and 108 controls (mean ± SD age in years for ASD/controls = 20.
8 ± 8. 1/ 19. 2 ± 7. 1). Quality indices for structural and functional MRI data did not differ between ASD and controls (two-tailed Student's t-test: p > 0. 27, t = 1. 08 for cortical surface extraction, p > 0. 28, t = 1. 07 for head motion). Details on subject inclusion and quality control are provided in the Methods and Supplementary Figure  1. Details on the independent replication dataset can be found in the Methods. Altered macroscale gradients in autism. We applied diffusion map embedding  26  , an unsupervised non-linear dimensionality reduction algorithm, to cortex-wide functional connectomes derived from resting-state fMRI in each individual. The first principal gradient explained 24% of connectome variance in our dataset (with similar variance explained in ASD and controls across gradients, two-tailed Student's t-test: t = 0.
Finally, the replication dataset showed virtually identical results (Supplementary Figure  7 ), confirming delayed SFC to the DMN but early acceleration in low-level networks in ASD. To integrate gradient and SFC findings, we presented SFC trajectories in a coordinate system spanned by the first two gradients (Fig. 3 ), a synoptic representation of the connectome hierarchy as suggested in the previous studies  8, 17. In controls, SFC consistently evolved from sensory and somatomotor networks situated at the bottom poles towards DMN core nodes at the top, following an almost straight line (Fig. 3c left ). Conversely, and despite an overall contracted arrangement in ASD (i. e. , lower principal gradient locations of the DMN, higher locations of sensory/sensorimotor networks), SFC transitions never fully converged at the top DMN regions (Fig.
Indeed, our connectome gradient mapping showed that both groups revealed an axis of connectivity variations with low-level sensory systems and transmodal DMN on two opposite ends and remaining networks in-between. However, in ASD this gradient was globally contracted compared to the neurotypical group. Consistent with these results, the complementary analysis of SFC also demonstrated that in ASD sensory-driven connectivity transition do not converge to transmodal areas. Further graph theoretical stratification of the results revealed hierarchical imbalances related to a failure to selectively activate the rich-club core, suggesting the reduced capacity of the network backbone to switch into a global and long-range network communication pattern. Collectively, our multi-method approach provides converging evidence for atypical connectome hierarchy as a system-level substrate of ASD, a finding that could be replicated across distinct datasets, parameter choices, and when controlling for motion and various methodological confounds.
Interestingly, separately analyzing children and adults in our dataset revealed disruptions in early systems in both, while DMN anomalies were more marked in adults with the condition. Please note that, however, the cross-sectional nature of our sample essentially prevents us from specifying true developmental roles of sensory and transmodal system atypicality in ASD. Future work, therefore, requires to be conducted based on more systemically controlled samples such as a longitudinal data of individuals with autism, often underrepresented from early childhood  24. We close by noting that our findings of dysfunctional macroscale integration and segregation in ASD was made possible by the open ABIDE data repository. These initiatives offer the neuroimaging and network neuroscience communities an unprecedented access to large datasets for the investigation of healthy and diseased brains.
As they have also highlighted variability in data quality across sites, our work involved an extensive case screening, several statistical corrections, and validation experiments to confirm the robustness of our findings. Specifically, we verified our results in variations of MRI quality and motion, given that such confounds have been suggested to contribute to the marked heterogeneity in previous results in ASD  15. In contrast to homogenized multisite data, heterogeneous datasets such as ABIDE may pose unique constraints on the accuracy of predictive routines that operate from one site to another. Indeed, a leave-one-site-out learner did not achieve as high accuracy as a 5-fold cross-validation learner that pooled data from sites. Notably, the employed 5-fold cross-validation nevertheless provided a more thorough control on classifier accuracy compared to commonly applied within-sample regression or leave-one-out cross-validation.
Methods Participants. We studied two datasets (i. e. , discovery and replication) selected from each wave of the openly-shared Autism Brain Imaging Data Exchange initiative (ABIDEI and II; http://fcon_1000. projects. nitrc. org/indi/abide)  23, 24. The discovery data originated from a previously described ABIDE-I subsample  23, 24. Briefly, we selected those sites that included children and adults with ASD and typical controls, with ≥10 individuals/group (n = 287, 143 ASD, 144 healthy controls). We restricted the analysis to males given the low prevalence of females in ABIDE-I. Detailed quality control included only cases with acceptable T1-weighted MRI, surface-extraction and head motion in rs-fMRI (see below), resulting in 211 individuals from three sites: The replication data was a subsample of ABIDE-II  24.
Inclusion criteria were similar as for the Discovery data, but also included females given an increased female ratio in ABIDE-II. We initially selected 177 individuals from three sites. After quality control as for the Discovery dataset, we included 103 individuals: Individuals with ASD underwent structured or unstructured in-person interview and had diagnosis of Autistic, Asperger's, or Pervasive Developmental Disorder Not-Otherwise-Specified established by expert clinical opinion aided by 'gold standard' diagnostics: Autism Diagnostic Observation Schedule, ADOS  25  and/or Autism Diagnostic Interview-Revised, ADI-R. These focus on three domains including reciprocal social interactions, communication and language, and restricted/repeated behaviors. Full scale/performance/verbal IQ was measured via WASI, WAIS III, and/or WISC III.