dataesr/scientific-paragraphs-categorization huggingface dataset:
Full Correlation Matrix Analysis (FCMA) The Problem: Computationally Tractable Analysis of the Complete Functional Connectivity Matrix Functional connectivity (FC) refers to coupling of activity in different regions of the brain; it is typically measured as the temporal correlation of BOLD activity across voxels. To assess FC in an unbiased way over the entire brain would require calculating the correlation across all pairs of voxels. However, given the number of voxels in most datasets, this is computationally challenging and results in data with very high dimensionality that are hard to analyze or interpret  [47]. To address this, traditional analyses of FC have restricted the number of voxel correlations, either by using one or a small number of "seed" regions (pre-selected sets of voxels with which all others are correlated; akin to ROIs in standard analyses) or by parcellating the brain into larger regions  [48]  and then correlating the mean activities of the parcels.
FCMA calculates the full correlation matrix at the voxel level, that is, the correlation of every voxel with every other voxel for any given set of time windows in a dataset. In multi-condition, multi-subject datasets, this is a massive computation: for example, a typical dataset with ∼30,000 voxels has ∼450,000,000 voxel pairs. The computational load only grows if the correlation matrix is computed for multiple time windows, as is often the case. To make this more tractable, FCMA leverages several optimizations, including high-performance kernels to calculate and classify correlations and Message Passing Interface (MPI)  [51]  to distribute the parallelizable tasks among multiple compute nodes. These optimizations make it possible to use the full correlation matrix computation in offline analysis and also in circumstances that require rapid calculation of FC data (e.
BrainIAK's reprsimul. brsa module contains two models, BRSA and GBRSA. The BRSA model follows the algorithm in  [57] , with the improvement that it also models spatial noise correlation. In addition to modeling spatial noise correlation, the GBRSA model marginalizes voxel-wise parameters such as signal-to-noise ratio and temporal auto-correlation coefficients of noise, and can estimate similarity structure from either a single participant or from a group of participants. The notebook accompanying this paper illustrates the usage of GBRSA on a group of simulated participants. Readers can easily adapt the example to the case of a single participant by providing only one participant's data to the model. The notebook also illustrates additional functions of the model: decoding task-related signals from new data and cross-validating the fitted model to left-out data.
• We can fit the model on a TR x Voxel dataset by iteratively alternating between estimating event patterns and estimating TR probabilities for each event. The number of events must be pre-specified, but cross-validation can be performed to determine the optimal number. This approach has previously elucidated how event structure can be represented at multiple timescales  [16]. • If we already know the event-specific patterns (e. g. , from an independent task), we can create a model with these patterns and infer event probabilities on a TR x Voxel dataset of neural responses. • If there are multiple datasets (with aligned voxels but different numbers of TRs) that share the same event sequence (e. g. , responses to different versions of the same narrative, or to perception and recall of the same narrative), we can find shared event patterns across datasets and the per-TR event probabilities for each dataset.
The Notebook The corresponding notebook demonstrates how to fit the HMM to real movie-watching data, align neural event boundaries with annotations, and apply the HMM to recall data  [16]. Note that the eventseg package includes two extensions beyond the original paper  [16] : • You can define multiple "chains" of events rather than a single sequence. For example, if subjects recalled one of multiple stories, a separate event chain could be defined for each story and the model will assume that recall is equally likely within any separate chain. • You can perform a more exhaustive fitting procedure when estimating the event patterns. This (slower) approach attempts to split events or merge neighboring events for better allocation throughout the time series.
Topographic Factor Analysis (TFA) The Problem: Efficiently Describing Network Structure As neural datasets are often large, studying network patterns that require huge (O(n 2 )) time and space to compute can be intractable. One way to address this is the approach used by FCMA (i. e. , using optimized computations to obtain the entire voxel × voxel correlation matrix). However, these full-connectivity patterns (because of their size) can be challenging to work with in downstream analyses. Further, summarizing patterns of correlations often requires additional analyses whereby voxels are thresholded and/or grouped into spatially contiguous clusters or regions of interest.
The Solution TFA  [64, 65]  takes a different approach, exploiting the strong spatial correlations in fMRI data [e. g. , 66] to derive a lower-dimensional description of the data that lends itself to efficiently characterizing full-brain connectivity patterns. Given a time series of 3D fMRI volumes, TFA finds a basis set of spherical "nodes" placed throughout the brain; each of these nodes represents a contiguous set of voxels (Figure  6A ). (Non-spherical regions may be approximated using multiple spherical nodes. ) Each brain image may then be described as a weighted sum of the images for each node (Figure  6B ). When multi-subject data are available, the locations and sizes of the nodes are constrained to be similar across people (Figure  6C ). Applying TFA to a multi-subject fMRI dataset yields a "ball and stick" representation of its underlying network dynamics (Figure  6D ).
TFA works by defining a generative model for fMRI data. According to the model, data are generated by first choosing an appropriate number of nodes, K, and then assigning their locations and sizes within a global template that parameterizes a model of the "prototypical" participant (Figure  6C ). Next, each individual participant's nodes are selected by adding noise to the global template's nodes (Figure  6C ). In this way, this global template serves as a prior for the perparticipant models, thereby ensuring that different participants' nodes share similar locations and sizes. Finally, TFA assigns per-timepoint activations to each node (Figure  6B ). The fMRI volumes are generated by sampling the node activation patterns at the voxel sampling resolution of the images (Figure  6A ). Applying TFA to an fMRI dataset entails "reversing" this generative process: given the fMRI data from each participant, the goal is to discover the most probable number of nodes, as well as the node locations and sizes, for each individual participant.
fmrisim The Problem: Simulating Realistic fMRI Data Methods for analyzing fMRI data have blossomed in recent years, yet there is a concurrent need to understand how best to use these methods. Simulations of fMRI data can aid in both the evaluation of complex designs and the analysis of data. Software packages have been created that offer flexible simulation of fMRI data  [75] [76] [77] ; however, no package was designed explicitly for simulating data for multivariate analyses. Moreover, no available packages can generate simulated data with noise properties that are matched to an existing fMRI dataset.
The Solution To fill in this gap, we developed fmrisim  [23] , an open-source Python package for simulating realistic fMRI data. fmrisim linearly combines a number of noise sources, inspired by biology and MRI physics, that are tuned in a data-driven fashion to match specific fMRI data that is provided as an input. Through an iterative fitting procedure, the noise properties of the simulation are updated to optimize the match of the simulated data to the real data (Figure  8 ). We previously validated that this fitting procedure produces accurate simulations of real data  [23]. We have used fmrisim to evaluate the power of different experimental design parameters  [23] , and also to evaluate the efficacy of new analysis methods  [78, 79]. fmrisim can be utilized in two main ways by researchers.
First, it can be used to explore and optimize different experimental design parameters and analysis pipelines. This is particularly valuable in the case of complex, multivariate designs where traditional methods for evaluating design efficiency  [80]  may be inappropriate. Second, fmrisim can be used to pre-register an experiment design and analysis pipeline to conduct confirmatory hypothesis testing. By establishing an analysis pipeline before any data is collected, simulation can be used as a sandbox to tune the analysis pipeline for testing a specific hypothesis, without compromising any real data. Hence, fmrisim  offers a unique opportunity to conduct explicitly confirmatory research with fMRI. Considering these use cases together, we believe fmrisim is a valuable tool to help researchers conduct more reproducible fMRI research.
The Notebook The corresponding notebook for fmrisim illustrates the simulation of a dataset and how it can be used for analysis. The hope is that this can be used as a template for simulating your own study. This notebook takes in an example functional dataset and simulates new data with the same noise properties as this real data. It performs each step of noise simulation individually, in order to give the reader a sense of what is being done. Signal is then inserted into the data. The signal is a multivariate pattern of voxel activity evoked by events from different fictious conditions. A classification analysis is then performed to evaluate these block differences. 29.
Topological Data Analysis (TDA) and Geometrical Analysis Innovations in TDA have generated remarkable new insights in neural coding  [81] [82] [83]. The BrainIAK Extras repository provides a wrapper for PHAT  [84] , a C++ library for high-performance persistent homology. Future extensions may include wrappers for Rivet  [85] , a C++ package for multiparameter persistent homology. Alongside TDA, geometric methods are starting to gain traction in neuroscience data analysis  [83, 86] , specifically as a tool to study how entangled and disentangled feature dimensions interact to determine neural codes across multiple contexts  [87, 88]. Lightweight implementations of these methods are currently being developed in BrainIAK for investigating context-dependent cognitive feature representations. These implementations will include several practically-motivated techniques to address problems associated with large and/or incomplete datasets, as well as diagnostic tools for cross-validation of findings.
Matrix-normal models Many models for fMRI analysis are framed as linear regression or factor models with gaussian noise. This includes variants of SRM, RSA, TFA, and ISFC (all discussed earlier in this article), the conventional fMRI generalized linear model (GLM), and others. Typically these models assume independently normally-distributed residuals in either the spatial or temporal dimension (and often both). To match the data to these independence assumptions, traditional approaches often preprocess their data to remove spatiotemporal correlations altogether. However, if the preprocessing model is misspecified (which is always true to some extent), fully removing these correlations removes some signal alongside the noise. In contrast, structured-residual models, herein called matrix normal (MN) models, choose to jointly model the "signal" alongside the "noise" or residual covariance, letting the same model apportion signal relative to noise.
The SaaS model also allows for remote testing and configuration. Unlike a typical SaaS that uses a specific cloud service, our framework allows users to run RT-fMRI data analysis on their choice of system (including their institution's own computing cluster), which can help users meet regulatory requirements. The combination of using cloud computing and the SaaS model in our framework has the potential to facilitate growth in the field (2) The ProjectInterface, which wraps the experimenter's code, processes the DICOM data and runs the experimenter's analysis code to obtain a measure of the participant's brain state. The experimenter accesses the cloud application from a browser page that can run on a laptop. Among other things, the experimenter can initiate/finalize the session, change settings, and even observe the graph output of the analysis results.
Tutorials The Problem: Learning Advanced fMRI Analysis It is usually quite challenging for a new user to learn advanced fMRI analysis. There are three main challenges in applying BrainIAK methods. First, one needs to learn Python, a language that has only recently gained traction in psychology and neuroscience departments. Second, the analyses require knowledge of machine learning techniques that may be unfamiliar to cognitive neuroscientists. Third, these methods need to be executed on high performance compute clusters, a task that is non-trivial for even advanced practitioners.
The Solution To overcome the challenges of applying BrainIAK methods, we have created a set of tutorials for advanced fMRI analysis (https://brainiak. org/tutorials) that are user-friendly, free to use, and opensource  [22]. These tutorials complement other learning resources that have become widely available: The BrainIAK tutorials cover 13 topics in fMRI analysis: setup, data handling, classification  [1] , dimensionality reduction, classifier optimization, RSA  [2, 26]  searchlight  [101] , seed-based connectivity, FCMA  [50] , ISC  [10]  and ISFC  [11] , SRM  [13] , event segmentation  [16] , and real-time analysis  [94]. Each tutorial is a Jupyter notebook  [102]  that provides a step-by-step introduction to one method. The materials are designed for individuals with only basic knowledge of fMRI, cognitive neuroscience, and Python coding.
Hence, detailed background information is provided for each method and dataset to introduce novices to the material. We show users how to avoid pitfalls like circular inference  [103] , and handle complexities that arise when working with large datasets. Moreover, the code is commented to support learning. Exercises are provided to encourage a deeper understanding of the methods. The methods are integrated with Nilearn functions for loading and manipulating neuroimaging data, and machine learning functions from scikit-learn. The tutorials are run on preprocessed datasets taken from experiments published in the literature. These datasets include block-design experiments, naturalistic movies, people listening to stories, and simulated datesets for real-time analysis. To scaffold the transition to using a high-performance computer cluster in order to analyze data at scale, we also explain how to use a SLURM scheduler and how to estimate memory and execution time for jobs, and we provide batch scripts to help the learner acquire this complex skillset.
Optimizations and Scalability Several of the methods in BrainIAK incorporate algebraic, algorithmic, and computing optimizations that make it possible to scale the analyses to high-resolution images and large numbers of subjects. Furthermore, BrainIAK uses parallel and distributed processing via Python multiprocessing as well as standard HPC technology (OpenMP and MPI). Searchlight analysis is a good target for a scalable implementation, since all searchlights are independent and can hence be executed in parallel. Searchlight analysis performs a separate configurable analysis for every voxel in the brain by passing a moving window over the brain, and limiting consideration to only local voxels at each step  [101]. Our implementation automatically parallelizes over the cores available in the CPU using Python multiprocessing, even when running on a laptop.
The same code can be run unmodified on an HPC cluster and can run in parallel over cluster nodes if launched as a set of MPI processes. Input data are automatically distributed in the cluster from the first MPI process and output is automatically collected. The data distribution across processors can be done by either distributing each subject to a different processor or splitting sets of voxels to different processors. Finally, to help non-experts take advantage of cluster execution, we provide a comprehensive tutorial that includes scaling advice, e. g. , estimating memory requirements (see tutorial: https://brainiak. org/tutorials/07-searchlight/). In FCMA  [52] , the most computationally-intensive part of the analysis involves a three-stage pipeline: correlation computation, within-subject normalization, and voxelwise SVM cross-validation.
To optimize for modern processors, we reduce the computation of Pearson correlation between voxel pairs to the multiplication of a voxel-by-time matrix and its transpose, by normalizing the data within each time epoch. Although many libraries such as Intel MKL have been optimized extensively for matrix multiplications, they do not perform well for whole-brain fMRI datasets, which typically involve tall-skinny matrices with large numbers of voxels and few timepoints. We employ several special optimizations for modern processors. First, we partition tall-skinny matrices into block matrices to fit the small amount of memory built into the CPU (L2 cache) for each thread. Second, the cache contents are retained across stages of the procedure pipeline by merging (fusing) two computation stages. In other words, when the current stage finishes the computations of a blocked matrix, it proceeds with the next stage computation of this block without waiting for other blocks of the current stage to complete.
Third, we carefully design data structures and workflow for vectorization. Such optimized single-node code runs 1. 5x-2. 5x faster than using Intel MKL and LibSVM libraries on Intel Xeon processors and 5x-16x faster than that on Xeon-phi processors. The parallel implementation of FCMA for computer clusters achieves near linear speedups. In SRM  [42] , the initial formulation of the problem requires the inversion of a square matrix whose size is the number of voxels by the number of subjects. We use the matrix inversion lemma and other linear algebra transformations to require only memory proportional to the number of SRM features, much lower than the number of voxels. We support parallelizing the computation by subject, using MPI. Finally, we minimize the data sent between MPI processes, which may otherwise become a bottleneck when running on an HPC cluster.
With the FastSRM algorithm  [43] , we can apply SRM to large datasets that do not fit in memory. Its efficient implementation (relying on an intermediate atlas-based representation) yields similar performance to the initial formulation while being faster and more memory efficient. In hierarchical TFA  [42] , the main bottlenecks we dealt with were large memory requirements for storing a certain Jacobian matrix and a large number of matrix inversions computed by an unconstrained non-linear least squares solver. To address the memory issue, we partition the model variables that determine the matrix size into two blocks; this doubles the required computations, but we consider this tradeoff to be worthwhile. To reduce the number of inversions, we use a constrained solver and apply the matrix inversion lemma.
Contributing to BrainIAK BrainIAK follows open collaboration principles. While the Princeton Neuroscience Institute and Intel Labs started the project, contributions are welcome from anyone. Contributions can take many forms: Python code for one of the analysis methods, C++ code for speeding up computation, Jupyter notebooks to showcase method usage, documentation, bug descriptions, or community interaction via email and chat. To encourage new contributors, we advertise simple tasks on our public issue tracker on GitHub. Source code is published on GitHub under an open source license (Apache 2. 0, except for the brainiak extras package, which uses LGPL 3. 0 because of its dependencies). The criteria for accepting new code are documented and automatically verified as much as possible. Documentation is essential for attracting users and contributors.
Therefore, we require each code contribution to be accompanied by documentation. Furthermore, we provide both simple examples and comprehensive tutorials for most of the methods. Contributions improving documentation are welcome. We have a public email list and a public chat room for community discussions. We try to provide a welcoming environment for anyone to discuss issues less formally than via GitHub. We found hackathons to be an effective way to attract new contributors. In addition to BrainIAK-specific hackathons we organized, we also submitted BrainIAK topics for larger hackathons, such as the one organized by OHBM.
Other parameters determine the spatial and temporal properties of noise (and spontaneous activity). Arrows indicate causal relations in a probabilistic graphical model. B. (G)BRSA marginalizes out intermediate variables that contribute to fMRI data, making it possible to compute the log likelihood of the fMRI data Y given the covariance structure U (the arrow with dashed contour); the algorithm then finds U that maximizes this log likelihood (the solid arrow), which can be converted to a similarity matrix of activation patterns. C. BRSA significantly reduces bias (spurious similarity structure) compared to traditional RSA on a simulated dataset with 16 task conditions. The four figures are (from top to bottom) the ground truth similarity structure in the simulated data, similarity structure recovered by BRSA and traditional RSA, respectively, and the theoretically derived spurious structure arising from the interaction between fMRI noise and the design matrix.
Acknowledgements Funding for this project was provided by  Intel Labs  (https://www. intel. com/intellabs) to J. D. C. , K. L. , P. J. R. , N. B. T. -B. , and  K. A. N , and also by the  John Templeton Foundation  to J. D. C. , N. B. T. -B. , and K. A. N. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the funders. We would like to thank the following people for help with many aspects of this project:  David Turner  for help with high-performance computing;  Benjamin Singer  for help with software installations and data management;  Ed Clayton  for organizing logistics for the workshops and hackathons.
We thus introduce the problem of clustering a sequence graph. We first propose two pairwise distance measures for sequence graphs, one based on edit distance and shortest path distance and another one based on SimRank. We then formalize the problem under each measure, showing also that it is NP-hard. In addition, we design a polynomial-time 2-approximation algorithm, as well as a heuristic for the problem. Experiments using real datasets and a case study demonstrate the effectiveness and efficiency of our methods.
3. We compare our algorithms against four state-of-the-art attributed graph clustering algorithms  [12, [16] [17] [18] , using two datasets from trust-aware e-commerce websites. Our results show that, unlike the competitors, our algorithms create high-quality clusters containing structurally similar nodes with similar strings (e. g. , users with trust bonds who are similar with respect to their history of reviewed products). Our results also show that the proxy measures we introduce do not substantially affect quality, while greatly improving runtime efficiency. For instance, our algorithm that utilizes the proxy of the product metric was up to 8 times faster than the most efficient competitor.