dataesr/scientific-paragraphs-categorization huggingface dataset:
Friends Dataset The dataset used in this study is a subset of the Courtois Neuromod Dataset, called Friends, where the participants have been watching the entirety of seasons 1 to 4 of the TV show Friends. This subset was selected because it provided a rich naturalistic soundtrack, with both a massive quantity of stimuli and a relative homogeneity in the nature of these stimuli, as the main characters of the series remain the same throughout all seasons. Subjects watched each episode cut in two segments (a/b) to allow more flexible scanning and give participants opportunities for breaks. There is a small overlap between the segments to allow participants to catch up with the storyline. The episodes were retransmitted using an Epson Powerlite L615U projector that casted the video through a waveguide onto a blank screen located in the MRI room, visible to the participants via mirror attached to the head coil.
Data acquisition The participants have been scanned using a Siemens Prisma Fit 3 Tesla, equipped with a 2-channel transmit body coil and a 64-channel receive head/neck coil. Functional MRI data was acquired using an accelerated simultaneous multi-slice, gradient echo-planar imaging sequence  (Xu et al. , 2013)  : slice acceleration factor = 4, TR = 1. 49 s, TE = 37 ms, flip angle = 52 degrees, voxel size = 2 mm isotropic, 60 slices, acquisition matrix 96x96. In each session, a short acquisition (3 volumes) with reversed phase encoding direction was run to allow retrospective correction of B0 field inhomogeneity-induced distortion.
Preprocessing of the fMRI data All fMRI data were preprocessed using the fMRIprep pipeline version 20. 2. 5 ("long-term support")  (Esteban et al. , 2019) , see Annex A for details. We used a volume-based spatial normalisation to standard space (MNI152NLin2009cAsym). The anatomical mask derived from the data preprocessing phase was used to identify and select brain voxels. Voxel-level 2D data matrices (TR x voxels) were generated from 4-dimensional fMRI volumes using the NiftiMasker tool from Nilearn  (Abraham et al. , 2014)  and a mask of the bilateral superior temporal gyri middle (middle STG), specifically parcel 153 and 154 of the MIST parcellation  (Urchs et al. , 2019) , resulting in 556 voxels. ROI-level 2D data matrices (TR x ROI) were generated from 4-dimensional fMRI volumes using the NiftiLabelsMasker tool from Nilearn with the MIST parcellation.
Overview The general approach we adopted to train encoding models of auditory activity relied on transfer learning and fine-tuning of a pretrained deep learning backbone. Audio waveforms were used as inputs to the backbone, generating a set of features as a function of time. Those features were then used to train a small downstream convolutional layer to predict the fMRI signal. We detail the backbone, fMRI encoding layer, hyperparameters and training procedures below. We used Pytorch and other python packages 3  , and trained our models on the Alliance Canada infrastructure on V100 and A100 GPUs.
Deep learning backbone The network we selected as our backbone is SoundNet, proposed by Aytar and collaborators  (Aytar et al. , 2016). We selected SoundNet for the following reasons : (1) SoundNet is fully convolutional, as all intermediate representations (i. e. layers) are obtained from 1D convolutions and pooling operators, (2) SoundNet was initially trained on a large dataset of natural videos from the Internet (Flickr), with a high degree of correspondence between the visual and audio content, (3) SoundNet obtained good performances on downstream auditory tasks using transfer learning, and (4) SoundNet was trained using a teacher-student training paradigm, in which the student network is fed with raw audio only, and is trained to minimise the discrepancy with probabilities extracted from images using pretrained networks.
The reason behind the choice of layer Conv7 output is that it has the same temporal resolution as the fMRI signal (0. 67 Hz). We implemented the SoundNet architecture followed by the fMRI encoding layer as a fully end-to-end trainable network, by adapting an open-source pytorch implementation of SoundNet 4. To train this architecture, we used AdamW  (Loshchilov & Hutter, 2019)  as an optimizer for the L2 regularisation and the weight decay, and we applied a learning rate scheduler that reduces the initial learning rate if no progress is achieved by the optimizer. MSE loss is used as our loss function to minimise the difference between the prediction and the actual fMRI signal. We added a padding to the kernels of the encoding layer (size : Kernel size -1). For training, we used the fMRI data from subjects watching the first three seasons of Friends.
For each subject, we used 75 percent of the dataset for only training, and left out 25 percent for validation. Season 4 was left out to be used only for testing. To evaluate the accuracy of the model's prediction, we computed the coefficient of determination R 2 between the prediction and the corresponding fMRI time series for each region or voxel, for the entirety of the selected dataset (training or testing). Table  1. Hyperparameters values explored for subjects' baseline model. The values selected for fine-tuning the models are marked in bold. For the kernel size and the learning rate, the range of values shifted by 1 unit between sub-06 and sub-01, as the optimal values for the previous subjects appeared to be the maximal value of the explored window. The subject order has been constrained by the availability of all four season fMRI data for each subject at the time of the analysis, as this study was taking place in parallel to the data collection.
As we are working on individual datasets, we wanted to explore the hyperparameters space for each subject's baseline models. However, doing so would be particularly time-consuming and highly costly in terms of computational resources. In order to constrain the hyperparameters grid search, we decided to explore all parameters only with the baseline model trained on sub-03 fMRI dataset. sub-03 has been chosen amongst others subjects as it was the subject showing the highest Signal-to-Noise Ratio (SNR) in their time series. With a range of different values explored for each of the different criteria (see Table  1 ), we explored 1296 possible configurations, for both whole-brain and middle STG models. As results were equivalent between both models, we decided to only keep results from Whole-brain models.
We then determined which value of each parameter was prevalent in the best cluster, by computing the statistical mode for the 20 configurations present in the cluster with the higher r 2 score. After deciding which parameters and values had the most impact on the capacities of the baseline model to predict fMRI signals from sub-03 dataset, we switched to other subjects, this time exploring only the most impactful hyperparameters on sub-03 and a limited set of values around the optimal value found for sub-03 (see Table  1 ). As a result, we explored only 27 configurations in the remaining subjects. For each subject, we defined the best hyperparameters values by computing the mode amongst the 10 most performing configurations. We did not use cluster analysis for other subjects, as we only had 27 configurations.
The left and right STGm had some of the highest r² scores (q < 0. 05 for all subjects). The posterior STG (STGp) also consistently ranked second in terms of prediction accuracy (see Supplementary Figure  1 ). Subject 05 was an exception, with the highest r² score in the left Middle Temporal Gyrus (MTG) superior, followed by the STG. Though the STG was best predicted, SoundNet also accurately predicted other auditory regions like the MTG and Heschl's gyrus in most subjects (Figure  1 ). This result supports our hypothesis that our baseline model can encode auditory processing from natural stimuli like movies, with some notable variations in performance between subjects, e. g. substantially higher performance was achieved in STG for sub-03 and sub-04.
SoundNet successfully encodes activity brain activity in the visual cortex and other regions Apart from the STG middle and posterior being the best ROI predicted, brain activity in other regions was also predicted by the models; for most subjects we observed ROIs in the visual cortex such as the Lateral Visual Network DorsoPosterior (LVISnet_DP) and the Ventral Visual Network (VVISnet), respectively scoring as high as 0. 12 and 0. 11 (max scores in sub-03). These ROIs proved to be the best predicted ROIs after the STG and the MTG in Sub-01, Sub-02, Sub-05 and Sub-06, revealing that our baseline models were also able to encode aspects of the processing of an audio stimulus outside of the auditory cortex. SoundNet successfully encodes high resolution brain activity in the Superior Temporal Gyrus We found SoundNet could predict fMRI signals from voxels in the middle STG for all subjects.
Are brain-aligned ANNs learning subject-specific representations? Do brain-aligned ANNs learn subject-specific representations? We found that models fine-tuned to individual subjects' fMRI data did not translate well to other subjects. Both whole-brain and STG models were more accurate when predicting data from their training subjects, with STG models showing higher individual specificity. This aligns with studies by Schwartz and collaborators  (Schwartz et al. , 2019)  and Rastegarnia and collaborators  (Rastegarnia et al. , 2022)  which also discovered idiosyncratic, subject-specific brain representations. Similarly, Porter and collaborators  (Porter et al. , 2023)  reported that individual models significantly outperformed a group model in a task-based decoding study. Our results echo these findings, suggesting a reliable and unique network organisation at the individual level in deep phenotyping datasets.
Limitations Our study was limited by its exclusive use of the SoundNet pretrained convolutional network. Other models we initially tested didn't yield satisfactory brain encoding results, hence our selection of SoundNet. However, these preliminary analyses lacked systematic hyper-parameter tuning and were conducted on varied datasets. Since our selection in 2019, AI and brain encoding research fields have advanced considerably, with more recent DNNs surpassing SoundNet's performance across all HEAR benchmark audio tasks. Though CNNs are still popular in brain encoding and decoding studies, particularly in vision due to convolution's similarity to early visual layer processing  (Seeliger et al. , 2021) , Natural Language Processing networks like BERT and transformers have shown promise  (Conwell et al.
Conclusions In our study, we developed the first set of auditory deep artificial networks fine-tuned to align with individual participants' brain activity. This was made possible by the Courtois NeuroMod project's massive individual data collection effort. We successfully fine-tuned a pretrained network called SoundNet to better encode individual participants' brain signals, showing varying degrees of improvement over a model that only adds an encoding layer to predict brain signals. These brain-aligned models improved performance on an extensive benchmark of downstream AI audio tasks. Individual brain models also did not generalise to other individuals, implying idiosyncrasies in either the learned representations or the brain mapping layer. These findings open a wealth of opportunities for future research, ranging from studying inter-individual variations to testing various model architectures and types of training data.
Acknowledgments The  Courtois  project on neural modelling was made possible by a generous donation from the  Courtois foundation , administered by the  Fondation Institut Gériatrie Montréal at CIUSSS du Centre-Sud-de-l'île-de-Montréal  and  University of Montreal. The  Courtois NeuroMod  team is based at " Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal" , with several other institutions involved. See the cNeuroMod documentation for an up-to-date list of contributors (https://docs. cneuromod. ca). The work was partly supported by a grant from the  Brittany region in France "Allocation de Recherche Doctorale " to NF, and a  Digital Alliance Canada  resource allocation grant to PB to access the Béluga high performance computing infrastructure. PB is supported by a salary award as senior fellow (chercheur boursier senior) of the " Fonds de Recherche du Québec -Santé ".
Artificial intelligence (AI) has emerged as a fundamental component of global agricultural research that is poised to impact on many aspects of plant science. In digital phenomics, AI is capable of learning intricate structure and patterns in large datasets. We provide a perspective and primer on AI applications to phenome research. We propose a novel human-centric explainable AI (X-AI) system architecture consisting of data architecture, technology infrastructure, and AI architecture design. We clarify the difference between post hoc models and 'interpretable by design' models. We include guidance for effectively using an interpretable by design model in phenomic analysis. We also provide directions to sources of tools and resources for making data analytics increasingly accessible. This primer is accompanied by an interactive online tutorial.
Driven mainly by the advent of machine learning (ML)a particular approach to AI in which intelligent systems learn and derive models from training datasetsand deep learning (DL)a specialized branch of ML that leverages neural networks Highlights Artificial intelligence (AI) presents unprecedented opportunities for computational data analytics in plant digital phenomics and will have an extraordinary impact on plant science in the coming decades. In this review we describe a humancentric explainable AI (X-AI) system architecture that is based on advancing novel and creative linkages between data science, plant science, and information systems that are oriented to multiple outcomes. We elucidate the mechanisms of X-AI by illustrating the difference between inherent explainability and post hoc explainability, and dispel misunderstandings that can dilute the importance of this crucial but understudied topic.
In wood anatomical images at a microscopic level, a mask region-based CNN (Mask R-CNN) was used to analyze the intrinsic variability of wood anatomical features in conifers, alder, beech, and oak  [21]. In analyzing multispectral and hyperspectral data, a CNN was used to provide explainable identification of biotic stress in individual soybean plants by incorporating saliency maps  [22] ; a PLS and a RF to estimate above ground biomass in maize  [23] ; an SVM, a KNN, and a linear discriminant analysis (LDA) to detect and segment root decay in wheat  [24] ; an LDA and a PLS to detect response to drought stress in bell pepper, courgette, sunflower, radish, foxtail millet, and sorghum  [25] ; and a data-mining sharpener to guarantee consistent spatial resolution among heterogeneous remote sensing image datasets to dissect the latent heat flux signature of poplar in response to drought  [26].
e. , paired data) perform better than a single source. For example, using a deep NN (DNN), the fusion of RGB, TIR, and multispectral data delivered superior performance over single-sensor data analytics for yield prediction in soybean  [32] ; similar results were reported for the estimation of soybean chlorophyll content, nitrogen concentration, leaf area index, and aboveground biomass by employing an extreme learning machine  [33]. Now and in the future, rigorous data integration of phenomics and other different omic datasets that were not originally set out to be integrated and are from distinct biological samples (i. e. , unpaired data) may help to dissect the biological mechanisms that underlie desirable traits and shed light on the flow of information that underpins plant responses to environmental stresses  [10, [34] [35] [36] [37].
Accordingly, the development Glossary Bias: systematic errors in the ability of artificial intelligence (AI) models to make correct predictions. Compute node: a backend node used for computing in a cluster and reached via a frontend node. Computer cluster: a group of interconnected computers working together as a single, integrated computing resource. Confusion matrix: a visual representation that describes the complete performance of an AI model, summarizing its predictions in four categories: true positives, true negatives, false positives, and false negatives. Crowdsourced: data collected by soliciting contributions from a large group of people rather than from traditional experiments. Explicit knowledge: the human knowledge that can be readily assembled and passed on by written or verbal instruction.
Trends in Plant Science of tools for capturing the complete set of metadata is poised to have high impact on the support of FAIR data. As standards and tools become more widely disseminated and explored, we envisage that metadata will become commonly annotated by users, expected by referees, and required by journals and data repositories. Furthermore, the workflows that are used to analyze data should themselves be FAIR  [43]. When data sharing is not viable because of possible privacy or security concerns, federated learning (FL; see Glossary) and communication-efficient FL offer an unprecedented opportunity to train AI models without sharing data  [44, 45]. FL has gained traction in medical imaging applications  [46] [47] [48]  and holds great promise for overcoming data sharing challenges in plant phenomics.
This primer provides suggestions on how to use AI effectively in plant phenomics, on how to ensure that human-centric X-AI can benefit all, and discusses various X-AI approaches and techniques. We have created a central directory of all publicly available plant imaging datasets, and report their sources, accessibility, and a summary of the species and organ systems represented (Table  1 ). This review is accompanied by an interactive tutorial to train an interpretable by design model to deliver predictive and prescriptive analytics to users. Our primer is intended as an educational resource for phenomicists who are interested in applying X-AI approaches and techniques, and for plant scientists who seek a high-level understanding of this rapidly evolving field. Data scientists and information systems (IS) scientists may also use this primer as an introduction to the promising applications of X-AI in phenomics.
AI and the bias cascade Multiple sources of bias can affect the performance of AI systems used in phenomics and can occur across the different development steps of AI applications: data collection or selection, data preprocessing, model development, model evaluation, and deployment. Introduced bias can have a domino effect as it propagates from its entry point to the succeeding development steps, creating a bias cascade. The bias cascade starts with the data collection or selection step, where experimental data are collected or selected from publicly available datasets (Table  1 ). Bias here can occur for several reasons: (i) 'measurement' bias, when the data contain faulty measurements originating from instrumentation malfunctions, wrong values from miscalibrated sensors, or errors of precision that result in data distortion  [51] ; (ii) 'label' bias, when the data are laden with subjective judgments of human experts and are thus inconsistently or wrongly labeled  [52] ; (iii) 'sample selection' bias, when the training data do not represent a random sample from the entire dataset  [53] , causing a model to ignore data belonging to classes that were not represented during the data selection Parallel computing: a form of computation in which multiple compute nodes operating simultaneously are used to solve a large problem broken into independent smaller parts that can be processed concurrently.
Trends in Plant Science process; and (iv) 'group attribution' bias, when a data sample is selected from an incorrect target population  [53] , and where a model can fail to distinguish between some classes and consider them to be the same. Data preprocessing is performed to eliminate noisy (e. g. , blurred images, images with unfavorable lighting conditions, images that do not represent the object of interest), incomplete (e. g. , unannotated images), and duplicate data, and to normalize datasets as needed to account for batch effects (e. g. , groups of images taken under different lighting conditions or with different A network of comprehensive automated weather stations collects hourly weather and soil data including, among others, rainfall, air temperature, solar radiation, relative humidity, and soil moisture and temperature.
Trends in Plant Science camera settings) or systematic experimental artifacts (e. g. , reflections in images). In this step, even if the training set is representative of the entire dataset, this can generate an intrinsically unbalanced dataset in which some plant species, genotypes, or even stresses are under-represented. Such cases can introduce the 'class imbalance' bias. Bias may also arise during the model development and evaluation steps, where a model is trained and its ability to generalize beyond the training set on new and previously unseen data are evaluated. Because most AI algorithms identify correlations between variables in the underlying data but without being able to detect causal relations, two biases are likely to arise: (i) the 'correlation fallacy' that confuses correlation with causation  [53]  where a model wrongly deduces a cause-and-effect relationship between correlated variables, and (ii) the 'apophenia' when a model sees patterns whereas none actually exists  [54].
g. , different weed or crop species), which is known as the 'domain shift' bias. Phenomic data can be structured, semistructured, or unstructured. Structured (e. g. , spreadsheet files) data (blue line) are typically 'at-rest', transformed into rows and columns, and loaded into relational databases in data warehouses using a process known as ETL. Semistructured (e. g. , extensible markup language files) and unstructured (e. g. , flat files) data (red line) are streamed 'in-motion', loaded into non-relational databases, and stored in data lakes in their raw form; their transformation occurs on-demand using a process known as ELT. Because ELT loads data immediately, it prevents any slowdown that often occurs at the transformation step, and thus enables near real-time analytics for fast and practical decision making.
When either ETL or ELT is used, data warehouses and data lakes store data as matrices, cubes, polytopes, or distributed in memory. A well-designed data architecture results in higher-quality phenomic datasets that allow plant scientists to ask biological questions and to devise data-driven analytics in the search for answers. Abbreviations: ELT, extract, load, transform; ETL, extract, transform, load. and Frontier at Oak Ridge National Laboratory) reach a billion billion (10 18 ) FLOPS and can deliver higher performance in pattern searching in phenomic big data, thus speeding up crop design. Quantum computers (e. g. , System One of International Business Machines and H1-2 of Quantinuum) represent a new paradigm in computation that leverages the fundamental principles of quantum mechanics to perform calculations.