dataesr/scientific-paragraphs-categorization huggingface dataset:
Long term Simply copying pixel data for each research project may not scale for imaging data, where storage could quickly become infeasible as the Safe Haven hosts ever greater numbers of studies each requiring large imaging datasets. An efficient method of sharing the pixel data between multiple studies may be required. However, each study will have different metadata, e. g. , study-specific patient identifiers in the image header, so a solution that combines shared pixel data with study-specific non-pixel metadata is needed. We plan to investigate different solutions such as a virtual file server (already developed in prototype), requiring each research group to purchase more disk space should their project require it, pulling images in batches/caching or another technical solution entirely.
To date, 869 "special case" rules have been added to the IsIdentifiable tool's dataset: e. g. , the "Princess Royal" hospital being identified as a name rather than an organization. r Huge number of images can make cohort creation cumber- some at the image level. To address this, we are adding support to the infrastructure and relational database to enable research coordinators to mostly operate at the study level. r By not anonymising each image once on initial receipt we in- troduce additional complexity and increased storage requirements. In addition, we have to repeat the anonymisation task each time the image is used, so we have more work to do. We trade this off against the ability to apply better anonymisation later. If the repetition proves an issue in real use, this can be mitigated through caching previously processed pixel data.
r Duplicating the image data for each research project will limit future scaling to multiple projects; some potential ways of addressing this are discussed in the previous section. r The relational database structure is not ideal for some more complex parts of DICOM. We believe that for cohort generation our flattened relational structure is simple and functional, but we may discover cases in the future where it becomes cumbersome for some parts of DICOM. If so, hierarchical data can be incorporated within MySQL via JSON columns.
Conclusions We have designed an architecture that meets the requirements of data governance and and initial indications suggest that it will manage and provide extracts of routinely collected imaging data linked to other relevant datasets for research from the >2 PB of SMI data. We have tested the extraction system on 5 use cases, based on real exemplar scenarios. The Background section of this article identified 5 challenges to using routinely collected clinical images for research. The limitations of existing patient-centric image handling, Challenge 1, are addressed by extracting and indexing other attributes identified by researchers: instead of retrieving a specific patient's imagery, we can search for images by a combination of parameters such as body part, patient age, or cross-referencing with other datasets, e.
g. , "all head/brain MRIs of patients with a diagnosis of glioblastoma. " To address Challenge 2, the pervasive presence of potentially identifying information within various data fields and the image pixels themselves, the platform uses a data controllerapproved, standardized de-identification workflow, with a "defence in depth" strategy employed: even if the anonymisation procedures fail earlier in the pipeline, unexpected information is detected and reported to the system operators before release to researchers. To mitigate the risks to the data controllers of providing unconsented routinely collected images for research the platform works in accordance with the Scottish Government guiding principles for secure linking, anonymising, and analysing datasets for research, where a subset of data for a specific cohort are linked for an approved research project and access provided via a Safe Haven environment (the NHS Scotland term for a Trusted Research Environment)  [41].
Access to the data can be revoked by the data controller at any time, and researchers cannot extract/output any information other than aggregate-level results from the environment. There is a separa-tion of the roles of indexer, linker (carried out by a trusted third party or Safe Haven staff), and researcher. The original identifiable data are maintained to enable linkage to other datasets (addressing Challenge 3, the barrier that anonymisation normally presents to such linking) but are not released for cohort building or for research. The use of encrypted numerical patient identifiers (using the same encryption for both image metadata and other eDRIS datasets) also facilitates some linkage without exposing the original identifier: an MRI scan can be matched with a cancer diagnosis or patient admission record without exposing that patient ID.
Developing the handling, indexing, and anonymisation system in a robust, reusable way and incorporating multi-layered safeguards allays data controller concerns about the quality of anonymisation (Challenge 4), while amortising the substantial development and testing costs across multiple research projects makes it more cost-effective to extract large sets of images for deep learning purposes, which might otherwise be uneconomical (Challenge 5). No non-trivial dataset is likely to be perfect, particularly when data gathered for one purpose is being re-purposed, so each research project will need to apply appropriate quality control checks; some will also bring new value, e. g. , an expert analysis of the images for a specific purpose. Each time an issue is identified by one project or new information added, this has the potential to improve the resources available to future projects.
In the absence of a pre-created perfect reference library, any project will have to choose between using a resource such as this, with the need for quality checks, and a much smaller but more discerning research-specific set if one is available or could be created. With CT scans in particular, the radiation dosage makes the reuse of existing imagery much more feasible. To access the SMI dataset for a research project, please contact eDRIS  [17]  in the first instance.
This work was supported by  Health Data Research UK , which receives its funding from  HDR UK Ltd  ( HDR-5012 ) funded by the  UK MRC, Engineering and Physical Sciences Research Council, Economic and Social Research Council, Department of Health and Social Care (England) ,  Chief Scientist Office of the Scottish Government Health and Social Care Directorates, Health and Social Care Research and Development Division (Welsh Government ),  Public Health Agency (Northern Ireland) ,  British Heart Foundation (BHF) , and the  Wellcome Trust.
Availability of Source Code and Requirements Snapshots of our code and other supporting data are openly available in the GigaScience repository, GigaDB: https://dx. doi. org/10. 5524/100780 Project name: Research Data Management Platform Project home page: https://github. com/HicServices/RDMP Registration: biotools:rdmp RRID:SCR 016268 Synthetic data generation (used in integration testing): https://github. com/HicServices/BadMedicine (EHR, biotools:badmedicine, RRID:SCR 018879) https://github. com/HicServices/BadMedicine. Dicom (DICOM data component for the above) Template building and relational database loading library: https://github. com/HicServices/DicomTypeTranslation (translation between DICOM types and database types, also includes example modality templates: biotools: dicomtypetranslation, RRID:SCR 018878) https://github.
Downloaded from https://academic. oup. com/gigascience/article/9/10/giaa095/5912826 by guest on 07 October 2020 [19]form has been implemented building upon the open source Research Data Management Platform (RDMP)[19]. The RDMP stores, manages, cleans, de-identifies, and processes data to create reproducible, auditable data extracts for research and in the past 5 years has been used to support >500 projects, generating >2,000 data extracts of mainly phenotypic text-based data for epidemiological research projects and clinical trials. The RDMP already provides many of the core components such as auditing, logging, deduplication, and anonymisation required for populating the relational database in a platform-agnostic way, as well as linkage and extraction; therefore it was efficient.
In particular, the methods presented in this work have two principal objectives: 1. To demonstrate that measures of functional connectivity can reliably be employed as features in machine learning models of brain age. To this end we build and validate models using three large open-source datasets: the Cambridge Center for Ageing and Neuroscience (CamCAN), the Human Connectome Project (HCP) and the ATR Wide-Age-Range datasets. 2. We further wish to interpret and inspect the proposed models in order to gain further insights into the changes in functional connectivity associated with ageing. This calls for the use of parsimonious and simple predictive models together with features whose relationship with functional conncetivity is clearly understood. Throughout this paper, we put forward the thesis that for the potential impact of functional connectivity assessment to be met (i.
are then used to predict brain age. Once the factor loadings are estimated as above, using one experimental data-set (we use CamCAN data in our experiments), we can then assess how these factors perform for brain age prediction on completely held-out data-sets; we demonstrate how the model generalizes well using HCP and ATR Wide-Age-Range datasets. To this end we proceed in a two-step framework. First, we seek to learn robust features which summarize properties of functional connectivity across a cohort of subjects in an unsupervised manner. Due to our focus on interpretability, we focus on linear latent variable models, such as principal component analysis (PCA) and its generalizations. The benefit of employing latent variable models such as PCA is that we may interpret the latent variables in terms of activity within functional connectivity networks, as proposed by  Leonardi et al.
Methods We focus our analysis on resting-state fMRI time series data which is collected across a cohort of N subjects. For the ith subject, it is assumed we have access to fMRI measurements over p fixed regions of interest, denoted by X (i) ∈ R p , as well as the subjects age, a (i) ∈ R +. Throughout this work we approximately model the fMRI data for each subject with a stationary multivariate Gaussian distribution, X (i) ∼ N (0, Σ (i) ), where Σ (i) denotes the covariance for subject i. Each entry in Σ (i) denotes the covariance between any pair of regions, which serves to define a measure of the functional connectivity  (Smith, 2012). As such, it follows that Σ (i) encodes a functional connectivity network over p regions where edges encode the marginal dependence structure. The goal of the proposed methods is to learn interpretable and robust models to predict the biological age, a (i) , of subjects given information relating only to their functional connectivity.
Hyper-parameter selection The proposed two-stage estimation framework requires the input of only one hyper-parameter: the dimensionality of latent variables k. In the context of PCA and factor analysis, this hyper-parameter directly corresponds to the number of principal components or factors inferred, and a wide literature exists for tuning such a parameter  (Jolliffe, 2011). One of the advantages of the latent variable models presented in Section 2. 1 is that they each correspond to probabilistic models whose likelihood can be directly evaluated. As such, a logical choice to tuning hyper-parameter k is to directly maximize the log-likelihood over held out data. In order to effectively perform hyper-parameter tuning as well as quantify the generalization performance of the proposed method, data was split into training, validation and test datasets as follows: • First, a subset of subjects were held out as test data.
As such, we obtain two datasets: ¶ X (i) 1:n , a (i) © i∈Strain and ¶ X (i) 1:n , a (i) © i∈Stest where S train , S test ⊂ {1,. , N } denote the non-overlapping sets of training and test subjects respectively. Recall N is the number of subjects present and we write X (i) 1:n to denote the n observations available for the ith subject. • Training data is further split into training and validation datasets on a subject-by-subject basis. Splitting the data in this manner allows for effective hyper-parameter tuning, using training and validation datasets, as well as for generalization performance to be measured using test dataset which corresponds to unseen subjects.
Experimental data The data employed in this manuscript corresponds to resting-state fMRI data taken from three distinct open-access repositories. There were small variations in the resting state functional MR image acquisition for each of the repositories considered: CamCAN  (Taylor et al. , 2015) , Human Connectome Project  (Van Essen et al. , 2013) , and the ATR Wide Age Range  (Ogawa et al. , 2018). The pre-processing employed on each dataset was as follows: • CamCAN: This dataset was pre-processed by us. Data was motion corrected, spatially smoothed with a 5mm FWHM Gaussian kernel, registered into MNI152 standard space using FLIRT  (Smith et al. , 2004 ) via a skull-stripped high-resolution T1 image and resampled to 4x4x4mm voxel sizes. Each high resolution T1 image was segmented into grey and white matter and cerebrospinal fluid using SPM Dartel  (Ashburner, 2009).
Mean timecourses for cerebrospinal fluid and white matter as well as 6 motion parameters were linearly filtered from each voxel to reduce non-neural noise. • HCP: We used the pre-processed resting state fMRI data from a random subset of healthy participants 1. Notably, the pipeline involved FIX ICA-based noise reduction process  (Salimi-Khorshidi et al. , 2014) , to remove individual sources of physiological, non-physiological and motion related noise. • ATR: We used the preprocessed data 2. The pre-processing pipeline notably included regressing out the global grey matter signal as well as signals from cerebrospinal fluid and white matter, to remove sources of spurious variation. All three pre-processed fMRI datasets were subsequently processed as follows: a cortical parcellation based on resting state functional connectivity analyses  (Power et al.
Results In this section we present a range of experimental results involving both synthetic and real resting-state fMRI datasets. Throughout this section, we contrast the performance of the various linear latent variable models presented in Section 2. 1. In particular, we study the performance across the following methods: factor analysis (FA), PCA, non-negative PCA  (Sigg and Buhmann, 2008) , MCF  (Hirayama et al. , 2016)  and MHA  (Monti and Hyvärinen, 2018). We first present results using synthetic data in Section 3. 1. These simulation experiments serve as a numerical validation of the proposed two-stage procedure. Experiments relating to brain age prediction from resting-state fMRI data are subsequently presented in Section 3. 2.
Synthetic data experiments In this section we evaluate the performance of the proposed two-stage estimation framework using synthetic data. To this end, we generate artificial data whose properties approximately match those which are frequently reported in fMRI studies. The objective is then to quantify which of the linear latent variable models presented in Section 2. 1 are able to both robustly recover the associated loading matrix, W , as well as learn the relevant factors which serve as accurate predictors of brain age on unseen subjects. Synthetic data was then generated in order to satisfy equations (1-2) and (  10 ). This is achieved as follows: • First, we randomly generated a factor loading matrix, W ∈ R p×k , which satisfied the constraints of both non-negativity and orthonormality.
Resting-state fMRI data experiments While the previous section presented results relating to synthetic data, here we present experimental results where the proposed two-step procedure is applied to three open-source resting-state fMRI datasets. The datasets considered correspond to the Cambridge Center for Ageing and Neuroscience (CamCAN) repository, the Human Connectome Project (HCP) repository, and the ATR Wide-Age-Range repository. The purpose of employing three distinct datasets is to effectively measure the generalization performance of the proposed approach on unseen data. As such, data from the HCP and Wide-Age-Range repositories was not employed during any of the model training and instead used exclusively as unseen test data. It is important to note that in addition to significant inter-subject variability  (Kelly et al.
CamCAN repository results Resting-state fMRI data was collected from a total of 647 subjects from the CamCAN repository. Subject ages ranged from 18 to 88 years of age (average age of 54. 31±18. 56, 318 males and 329 females). The CamCAN dataset was employed as the principal dataset in the proposed two-step procedure, implying that it was employed to learn both the functional network structure in the unsupervised learning stage and the linear regression models in the supervised learning stage. As such, the data was split into training, validation and test subsets as described in Section 2. 3. Step 1: unsupervised functional network inference The first stage of the proposed framework involves the estimation of reproducible functional connectivity networks via the use of the various linear latent variable models discussed in Section 2.
1. The number of functional networks inferred corresponds directly to the dimensionality of latent variables, which is determined by hyper-parameter k. As each linear latent variable model can be interpreted as a probabilistic model, we select hyper-parameter k by maximizing the log-likelihood over the validation dataset. This resulted in the choice of k = 5 when the loading matrix was restricted to be both non-negative and orthonormal, as proposed by  Hirayama et al. (2016)  and  Monti and Hyvärinen (2018). While it is possible that the choice of hyper-parameter may vary across distinct latent variable models (e. g. , for PCA or factor analysis), we choose to keep the choice of k fixed across all models as this facilitates model comparison and interpretation of results. The left panel of Figure  5  visualises the results when the MHA linear latent variable model was employed.
1 where employed to learn reproducible sub-networks parameterized by a loading matrix, W ∈ R p×k. The activity within each functional network, defined as in equation (  11 ), was subsequently employed as features to predict biological age using linear regression. We note that the CamCAN repository, as well as HCP and ATR repositories, each contained over a hundred subjects each. This is in contrast to typical fMRI studies, where the sample size is often in the range of 20 to 30 subjects  (Poldrack et al. , 2011 , Cremers et al. , 2017). Furthermore, recall that the goal of experiments presented are to quantify performance on unseen resting-state fMRI data with a view to providing an indication of how each of the linear latent variable models employed would perform in a typical fMRI study. As such, throughout the remainder of this section we report the performance, in terms of mean absolute error, over random subsets of 30 subjects from each repository.
As mentioned previously, the choice of k = 5 was selected in by maximizing log-likelihood over a validation dataset (i. e. , in an entirely unsupervised manner -data regarding subject ages was not considered). Figure  7  visualizes the performance on the unseen test dataset for the specific choice of k = 5, for all possible choices of linear latent variable models. The results indicate that as additional constraints are introduced to the loading matrix, the generalization capabilities of the models also improve. As such, MCF and MHA, which introduce the most stringent constraints corresponding to both non-negativity and orthonormality, obtain the best generalization performance. Thereafter non-negative PCA, which relaxes the requirement for orthonormality, is the next most competitive latent variable model.
Transfer onto HCP and ATR Wide-Age-Range repositories The results of Section 3. 2. 1 provide a measure of performance, in terms mean absolute error in predicted brain age, within a large-scale resting-state fMRI dataset. However, it is widely accepted that in addition subject-specific noise, there are several other significant contributors to noise in fMRI data: these include issues related to scanner noise and frequency of acquisition of images  (Friedman et al. , 2006 , Bennett and Miller, 2010 , Poldrack et al. , 2011). As a result, in order to thoroughly verify the generalization  We note that the introduction of non-negativity or orthogonality constraints leads to improved generalization (see Table  1 ). The number of functional networks was k = 5. performance of the proposed methods, we employ resting-state fMRI data from the HCP and ATR Wide-Age-Range repositories.
We note that data from the aforementioned repositories was employed only for testing purposes, as such it was not employed to learn the network structure across subjects, nor to tune the parameters of the linear regression models. To summarize, prediction of biological age on both the HCP and ATR Wide-Age-Range repositories was performed as follows: First, the loading matrix, Ŵ was employed to obtain estimated activity within each network, as detailed in equations (  11 ) and (  12 ). Subsequently, predictions of biological age were obtained using equation (10). At each stage both Ŵ and β are the parameters inferred using the CamCAN dataset (i. e. , there was no fine-tuning of parameters). As a result, performance on both HCP and ATR Wide-Age-Range datasets provide a robust measure of generalization performance to entirely unseen data.
Results on the HCP data are provided in Figure  8. As expected, the mean absolute errors are larger for each of the distinct latent variable models when compared to the results of on the camCAN dataset (Figure  7 ), which will be partially the result of varying scanner noise and image acquisition properties. Importantly we note that, as with the camCAN dataset, there once again a relationship between the introduction of additional constaints (in the form of non-negativity and orthonormality) and generalization performance. As before, methods such as PCA and factor analysis which do not introduce any constraints had the weakest performance as well as the largest drop in performance, see Table  1. Whereas the methods introducing both non-negativity and orthonormality yielded the best generalization performance as well as the smallest drop.
These results thereby serve as additional evidence that the introduction of robust constraints can serve to improve both the interpretability of results (as discussed in Section 3. 2. 1) as well as the predictive power of associated models. The HCP results presented above serve to partially validate the predictive models trained using the camCAN dataset. However, one significant limitation of the HCP dataset is that subject ages only range from 22 to 37 years of age. This is particularly relevant in the context of brain age biomarkers, as many neurodegenerative diseases of interest will be associated with advanced ages. As a result, we further validated the generalization capabilities of the proposed brain age prediction models on the ATR Wide-Age-Range dataset, which had subjects ranging from 20 to 70 years of age.