dataesr/scientific-paragraphs-categorization huggingface dataset:
We leverage graph mining-based sampling schemes and an effective subgraph decoding strategy to significantly lower the computational complexity of graph AE and VAE models, while preserving or even improving their performances. • We propose an in-depth theoretical and experimental analysis of our method. We demonstrate its empirical effectiveness on seven graphs, with various characteristics and natures, and with up to millions of nodes and edges. • We publicly release the code of FastGAE 3  , to ensure reproducibility and future usages. The remainder of this paper is organized as follows. After reviewing key notions on standard graph AE and VAE and on their complexity in Section 2, we present and analyze FastGAE, our scalable framework, in Section 3. We report our experimental evaluation and a discussion of our results in Section 4, and we conclude in Section 5.
Encoding the Entire Graph. As explained in Section 2. 3, GCN models  [3]  and their scalable extensions  [30, 31, 38, 40 ] can effectively process large graphs. Therefore, in our FastGAE framework, we rely on these models to encode all the nodes into the embedding space. More precisely, in the following experiments, we implement standard GCN encoders for the sake of simplicity and for an easier comparison to existing graph AE and VAE architectures. This design choice is made without loss of generality, the FastGAE framework being valid for any other encoder producing an embedding matrix Z.
1, FastGCN, GraphSAGE or Cluster-GCN could actually be used in conjuction with FastGAE, as encoders. Futhermore, FastGAE is also more elaborated than data cleaning methods that simply consist in removing some nodes from a graph, e. g. the low-degree ones, to reduce its size. Indeed, in the case of FastGAE with degree sampling, low-degree nodes are still 1) fully used in the GCN encoder, and 2) might also appear in some subgraphs that we decode (but less often than high-degree nodes). As we leverage new different subgraphs at each iteration, we explore different parts of the entire graph during training. Last, we note that effective subset selection for faster learning has already provided promising results in the machine learning community  [51, 52, 53] ; however, contrary to these works, we focus on an unsupervised graph-based problem, and our sampling methods remain fixed throughout learning as we rely on graph mining to select G (S).
Datasets We provide experimental results on seven graphs of increasing sizes. Their statistics are presented in Table  1. We first study the Cora, Citeseer and Pubmed citation networks  5  , with and without node features corresponding to f -dimensional bag-of-words vectors (with f = 1433, 3703 and 500 respectively). Nodes are clustered in respectively 6, 7 and 3 topic classes, acting as ground truth communities. These datasets are common benchmarks for evaluating graph AE and VAE (see Kipf and Welling  [11]  and a majority of recent works  [15, 16, 17, 18, 19, 26, 27, 28, 54] ). For these medium-size graphs, we Then, we consider four significantly larger graphs, with up to millions of nodes and edges, and for which training standard graph AE or VAE is intractable. We consider the Google 6  hyperlinks web graph, the Youtube 7  social network of users (friendship connections), the US Patent 6 citation network, and a synthetic graph, denoted SBM, generated from a stochastic block model which is a generative model for random graphs  [55].
• We also perform node clustering experiments, on datasets with ground truth communities. For this task, after training models on complete versions of the graphs, we run k-means algorithms in embedding spaces to cluster the z i vectors. We compare these clusters to the ground truth ones using the mean Adjusted Mutual Information (AMI) scores 9  on test sets. We emphasize that we chose to focus on these two tasks instead of a more direct reconstruction task (despite working with autoencoders), as AUC and AP scores from link prediction, as well as AMI scores from node clustering, are more insighful and understandable metrics than a direct reporting of some cross-entropy or ELBO reconstruction losses. Besides, we also aimed at providing a consistent experimental setting w. r. t. the existing literature on graph AE and and VAE that, for the most part, focused on link prediction and, to some extent, on node clustering tasks.
2. 2). We conjecture that current graph AE and VAE models might be suboptimal to effectively reconstruct communities in graph data ; this claim is consistent with recent experiments on these datasets  [18, 36]. As our objective, in this paper, was to scale existing graph AE/VAE, but not to ensure nor to claim their superiority over all other methods for node clustering, we do not further investigate this limit here. Nonetheless, future works on more effective cluster reconstruction from AE/VAE embeddings could definitely lead towards the improvement of these models.
Confined migration regulates co-accessibility of enhancer-promoter interactions One central mechanism by which chromatin accessibility changes can regulate gene expression is through enhancer-promoter interactions (EPIs) that exhibit co-accessibility, i. e. , having the same direction of accessibility changes  74, 75. To identify potential EPIs in "up" and "down" peaks, we assigned peaks that were within 1 kb upstream of the nearest gene's TSS to be "promoter-associated", and all other peaks as "nonpromoter-associated". As expected, there were many more promoter-associated peaks in "up" peaks than in "down" peaks (Suppl. Fig. 11a ). We then mapped all peaks to the DNase I hypersensitive regions in the ENCODE database  76  , as DNase I hypersensitivity validates chromatin regulatory elements, including promoters and enhancers  55.
"Up" peaks overlapped much more with the DNase I data, suggesting that "up" peaks contained more putative promoters and enhancers than "down" peaks (Suppl. Fig. 11b ). Based on the validated peaks, we found that EPI distance (promoter to non-promoter, peak-to-peak distance) was significantly longer for "up" peaks, compared to "down" peaks (Suppl. Fig. 11c ). This finding suggests that long-range (i. e. , more than a few hundreds of kb) enhancer-mediated regulation plays a role in "up" peaksassociated genes, which is a sign of epigenetic dysregulation and compromised chromatin domains in cancer  77. We identified 41 and 7 EPIs in "up" peaks and "down" peaks, respectively, based on the conventional 500 kb cutoff  78  (Suppl. Fig. 11d ). Representative EPIs in "up" peaks mapped to the enhancer and promoter database annotations (Suppl.
Extended imaging using an incubator microscope Long-term imaging was performed using an IncuCyte Live-Cell Analysis System (Sartorius) inside the cell culture incubator. Cells expressing NLS-GFP and migrating inside the microfluidic migration devices were imaged using the IncuCyte filter module for 5 days, every 10 minutes, with a 20× objective. The acquired migration movies were processed and exported using the IncuCyte ZOOM software. At the end of imaging, the cells were immediately fixed for IF staining of heterochromatin and euchromatin marks.
Confocal microscopy Fixed cells on coverslips and live cells migrating in microfluidic devices were imaged with an inverted Zeiss LSM700 confocal microscope. Z-stack images were collected using 20× air (NA = 0. 8) or 40× waterimmersion (NA = 1. 2) objectives. Airy units for all images were set at 1. 0. For overnight live cell imaging, the image acquisition was automated through ZEN (Zeiss) software at 10 minutes intervals, at 37°C in a heated stage chamber for 12-16 hours. Where indicated, temperature of the heated chamber was lowered to 33°C.
2 second intervals. The FRAP image acquisition was automated through ZEN (Zeiss) software. Fluorescence recovery was measured as percentage of recovery (with pre-bleach value as 100%), normalized to unbleached control. Immobile fractions were measured as fluorescence intensity that had not recovered after 24 seconds, after the recovery curve had reached plateau (by around 22 seconds). ATAC-seq was done using cells collected after 48 hours of migration in collagen matrices at specified collagen concentrations (low: 0. 3 mg/mL; medium: 1. 0 mg/mL; high: 1. 7 mg/mL), following Omni-ATAC protocol  56  with custom modifications. Scripts used to perform the analyses including exact parameters can be accessed at https://github. com/jaj256/ConfinedATAC. Please refer to Supplementary Information for detailed methods of ATAC-seq sample preparation, library preparation and analysis pipeline.
Image analysis Image sequences were analyzed using ZEN (Zeiss), ImageJ or MATLAB (Mathworks) using only linear intensity adjustments uniformly applied to the entire image region. Region-of-interest intensities were extracted using ZEN or ImageJ. All confocal image stacks were three-dimensionally reconstructed as maximum intensity projections, and then processed with standard background subtraction (rolling ball). To quantify normalized heterochromatin values, nuclear areas were selected by thresholding using the medianfiltered DAPI channel. Mean grey intensity of the heterochromatin mark (H3K9me3 or H3K27me3) and the euchromatin mark (H3K9ac) were quantified, and the heterochromatin intensity was divided by the euchromatin intensity to obtain normalized heterochromatin values. Apoptotic fragments and overlapped nuclei were manually excluded from the dataset.
Experiments with high variability of cell seeding between control and experimental groups were excluded. For all other quantification of staining intensities, nuclear areas were selected by thresholding using the median-filtered DAPI channel (or comparable nuclear signal when DAPI was not available), and mean grey intensities of channels of interests were quantified. All staining thresholding and quantification were performed through custom ImageJ Macros (available at https://github. com/chiehrenhsia/ConfinedMigration) to ensure consistency and reproducibility. For generation of normalized heterochromatin images, the heterochromatin mark channel (H3K9me3 or H3K27me3) was divided by the euchromatin mark (H3K9ac) channel using the "image calculator" function in ImageJ. In the euchromatin (H3K9ac) channel, pixels with 0 value were converted to 1 to ensure division without errors.
Cytoplasmic GFP signal was calculated by whole-cell GFP signal subtracting nuclear GFP signal. Nuc/Cyto ratio was then calculated by dividing nuclear signal with cytoplasmic signal. Nuc/Cyto ratio of GFP-HDAC3 from 2-6 time points immediately before and after nuclear transit were quantified and averaged. To quantify nuclear transit time through microfluidic migration channels, timelapse cell migration movies were analyzed using a custom MATLAB program, as described previously  93. Any single outlier with a transit time 180 minutes greater than the rest of the dataset was excluded in each experiment. Graphs were generated in Prism 7. 0a or 8. 2. 0 (GraphPad) or Excel (Microsoft), and figures were assembled in Illustrator (Adobe).
Acknowledgments The authors thank  Cornell University's Transcriptional Regulation & Expression Facility (TREx) , especially  Jen Grenier , for ATAC-seq library preparation their help with the analysis;  Cornell University's Biotechnology Resource Center (BRC) Genomics Facility  for ATAC-seq library quality control and sequencing;  Cornell University's Statistical Consulting Unit (CSCU)  for suggestions and reviewing of statistical analysis. The authors thank  Philipp Isermann ,  Tyler Kirby ,  Pragya Shah , and other members of the Lammerding group for their helpful discussions and support. This work was supported by awards from the  National Institutes of Health  ( R01 HL082792 ,  R01 GM137605 ,  U54 CA210184  to J. L. ), the  Department of Defense Breast Cancer Research Program  ( Breakthrough Award   BC150580  to J.
Data Availability ATAC-seq data that supports the findings of this study have been deposited in Gene Expression Omnibus (GEO) repository, with the accession number GSE181247. The other datasets generated during and/or analysed during the current study are available from the corresponding author (Jan Lammerding; jan. lammerding@cornell. edu) on reasonable request. The ImageJ Macro scripts used to analyze confocal immunofluorescence staining images and to generate normalized heterochromatin images can be accessed at https://github. com/chiehrenhsia/ConfinedMigration. The R scripts used to perform ATAC-seq analyses including the exact parameters can be accessed at https://github. com/jaj256/ConfinedATAC.
The triple-network model of psychopathology is a framework to explain the functional and structural neuroimaging phenotypes of psychiatric and neurological disorders. It describes the interactions within and between three distributed networks: the salience, default-mode, and central executive networks. These have been associated with brain disorder traits in patients. Homologous networks have been proposed in animal models, but their integration into a triple-network organization has not yet been determined. Using resting-state datasets, we demonstrate conserved spatio-temporal properties between triple-network elements in human, macaque, and mouse. The model predictions were also shown to apply in a mouse model for depression. To validate spatial homologies, we developed a data-driven approach to convert mouse brain maps into human standard coordinates.
To validate this, we generated humanized synthetic brain states derived from mouse brain maps and spatially compared them to matched human brain states. Beyond spatial homology, we hypothesize that transitions between brain states are conserved between the species examined, thus, defining temporal similarity. In this work, we establish shared spatio-temporal brain state properties between the human, macaque, and mouse brains. We test the predictions of the triple-model in mice by examining brain state occurrences in a dataset containing resting-state recordings in a mouse model for depression  [16]. We then generate a precise neuroanatomical model for the triple-network organization using a high-resolution mouse tracer dataset. Finally, we validate this anatomical model using functional stimulation of two key nodes of the salience network commonly implicated with depressive disorders, namely the dorsal raphe (DR) nucleus and the insular area.
METHOD Data and code availability The complete preprocessed mouse functional Magnetic Resonance Imaging (fMRI) datasets are freely available at https://doi. org/10. 34973/1he1-5c70 (resting-state) and https://doi. org/10. 34973/raa0-5z29 (optogenetics, CAPs). The S1200 human connectome dataset is available at https://db. humanconnectome. org/ [17]. The viral tracer data is available from https:// connectivity. brain-map. org/ [18]. The mouse template is available at https:// atlas. brain-map. org/. The complete code to reproduce the study is freely available as an R notebook at https://gitlab. socsci. ru. nl/preclinicalneuroimaging/insula. The repository also includes tables, figures, and imaging assets. The code to produce synthetic brain maps is available at https://gitlab. socsci. ru. nl/preclinical-neuroimaging/homologous-brain-expression-map.
Data processing and analysis Mouse and macaque data were preprocessed as follows. Anatomical images were corrected for B1 field (N4BiasFieldCorrection), denoised (DenoiseImage), brain-masked (antsBrainExtraction. sh) and registered using SyN diffeomorphic estimation (antsRegistration) to the AIBS template (mouse, https://atlas. brain-map. org/) or F99 template (macaque)  [19]  using ANTs  [20]. Functional images were despiked (3dDespike), motion-corrected (3dvolreg), corrected for B1 field, denoised, and brain-masked, before being registered linearly to their corresponding anatomical images. Resting-state fMRI (rsfMRI) data were band-pass filtered (3dBandpass, 0. 01-0. 25 Hz for the mouse, 0. 01-0. 10 Hz for the macaque). Optogenetic fMRI (ofMRI) data were high-pass filtered (0. 01 Hz). Independent component analysis (melodic) was applied, and nuisance components were determined using FIX before being regressed out (fslregfilt).
Brain state analysis Brain states were derived using co-activation patterns (CAPs). These were estimated using the TbCAPs toolbox (https://c4science. ch/source/ CAP_Toolbox. git) [21]. The anesthetized mouse dataset consisted of N baseline = 47, N CSS = 25, and N control = 27 scans acquired at 9. 4 T with repetition time = 1 s for 360 volumes  [16]. Mice were mechanically ventilated and anesthetized with 0. 5% isoflurane, together with 0. 05 mg/ kg medetomidine bolus and 0. 1 mg/kg infusion (see Optogenetics: imaging for detailed protocol). The awake mouse dataset consisted of N = 54 scans, acquired at 11. 75 T with repetition time = 1. 2 s for 180 volumes. The macaque dataset consisted of 10 male rhesus macaques (median age = 4. 98 years; median weight = 9. 25 kg) imaged at 3 T under isoflurane (1%) anesthesia and acquired with repetition time = 2.
28 s for 1600 volumes. The human rsfMRI dataset consisted of 15 humans imaged at 7 T in the awake condition with open eyes, sampled from the S1200 database  [17] , and was acquired with repetition time = 0. 78 s for 900 volumes. Volume selection was performed according to two criteria: (1) activity within at least one of three seed regions-the insula, anterior cingulate area (ACA), and the dorsolateral prefrontal cortex (human), frontal eye field (macaque), or primary motor area (mouse), and (2) framewise displacement <0. 5 mm for macaques and humans or 0. 2 mm for mice  [22]. This led to 80. 4% (min = 73. 9%, max = 85. 3%) retained volumes on average in anesthetized mice, but 35. 8% (min = 1,1%, max = 76. 7%) in awake mice due to increased movement. In macaques, the percentage of retained volumes was lower (mean = 19.
Synthetic brain maps Synthetic brain maps were generated using in situ transcriptomic similarity between available mouse in situ hybridization and human microarray data. A list of mouse and human homolog genes was obtained using Ensemble Biomart (http://www. ensembl. org) and cross-referenced to the list of genes available in the Allen Brain Atlas (https://mouse. brain-map. org/). The human data (https://human. brain-map. org/) was downloaded from the Allen Brain Atlas and processed using the automatic pipeline from the abagen (v. 0. 0. 7) package  [27]. In total, 3218 genes available in both 81 human brain regions-of-interest from the Desikan Killiany atlas  [28]  and mouse in situ whole-brain maps were retained. All genes were normalized to a range of 0-1 to account for amplitude differences.
Diffusion tensor imaging The S1200 7T diffusion tensor imaging dataset (N = 178) was reconstructed using 'bedpostx' with default parameters. Probabilistic tractography was performed using 'probtrackx', also with default parameters. Three seeds positioned on the left hemisphere were used: anterior insular area (MNI152 coordinates -36. 14, 10. 30, 2. 56), ACA (-6. 77, 45. 80, 21. 12), and dorsolateral prefrontal cortex  (-43. 83, 25. 41, 35. 44) , for the salience, default-mode, and central executive networks, respectively. Seed positions were informed from the co-activation pattern analysis. Gray-matter projections were extracted using 'fslmeants' and the Desikan Killiany atlas, and spatially compared to humanized synthetic brain maps.
Optogenetics: imaging Animal preparation was performed as described previously  [30]. Anesthesia was induced with 4% isoflurane; subsequently, the animals were endotracheally intubated, placed on an MRI-compatible cradle, and artificially ventilated (90 breaths/minute; Kent Scientific Corporation, Torrington, Connecticut, USA). A bolus with a mixture of medetomidine (Dormitor, Elanco, Greenfield, Indiana, USA) and Pancuronium Bromide (muscle relaxant, Sigma-Aldrich Pte Ltd, Singapore) was administered subcutaneously (0. 05 mg/kg), followed by a maintenance infusion (0. 1 mg/ kg/h) while isoflurane was simultaneously reduced and kept to 0. 5%. Care was taken to maintain the animals' temperature at 37 °C using a feedbackcontrolled water bath. The data were acquired on an 11. 75 T scanner (Bruker BioSpin MRI, Ettlingen, Germany) equipped with a B-GA09S gradient system, a 72 mm linear volume resonator coil for transmission, and a 10 mm single loop surface coil.
Images were acquired using Paravision 6. 0. 1 software. An anatomical reference scan was acquired using a spin-echo Turbo-RARE sequence: field of view (FOV) = 17 × 9 mm², FOV saturation slice masking non-brain regions, number of slices = 21, slice thickness = 0. 45 mm, slice gap = 0. 05 mm, matrix dimension (MD) = 200 × 100, repetition time (TR) = 2742 ms, echo time (TE) = 30 ms, RARE factor = 8, number of averages = 2. fMRI was acquired using a gradient-echo echoplanar imaging sequence with the same geometry as the anatomical scan: MD = 60 × 30, TR = 1000 ms, TE = 11. 7 ms, flip angle = 50°, volumes = 720, bandwidth = 119047 Hz. Field inhomogeneity was corrected using MAP-SHIM protocol. ChR2 photostimulation was provided through a blue light laser (473 nm, LaserCentury, Shanghai Laser & Optic Century Co.
Statistics and data representation Voxel-wise one-sample and two-sample t-tests were determined using fsl_glm, and cluster-corrected for multiple comparisons using the easythresh function, with a threshold set at either p = 0. 05 or 0. 001, as specified under the color bar. Linear models are estimated in R using the 'lm' function. Standardized coefficient parameters and 95% confidence intervals are estimated using the 'parameters' package for R  [32]. Group comparisons are represented using Gardner-Altman plots (https://www. estimationstats. com/#) [33]. The raw data are plotted on the left or upper axes; each mean difference is plotted on the right or lower axes as a bootstrap sampling distribution. Group differences are normalized to Cohen's d. These are depicted as dots; 95% confidence intervals are indicated by the ends of the vertical error bars.
RESULTS To compare resting brain activity patterns in the three species, we reduced resting-state functional Magnetic Resonance Imaging (fMRI) fluctuations acquired in human, macaque, and mouse recordings into brain states, the CAPs  [21]. The human data consisted of a sub-set (N = 15 subjects) of the 7 Tesla Human Connectome Project S1200 dataset  [17]. The macaque data (N = 10) was acquired in free-breathing anesthetized macaques  [35]. The mouse data (N = 47) consisted of the baseline scans of ventilated mice previously acquired  [16]. We clustered the images acquired in humans, macaques and mice into six brain states consisting of a spatial map and a temporal activation profile  [21, 36]. We opted for six states as this was the optimal clustering value for the mouse dataset, and was also reported as such, previously  [25].
Advantageously, such a representation of brain activity is data-driven, comprehensive, and comparisons can be made in absence of exact spatial homology. We observed matching spatial patterns across species (Figs. 1a, S1 ). The states mainly described co-(de)activation clusters overlapping with areas previously associated with the default-mode (#1, 2, 3), the salience (#5, 6), and the central executive network (#1, 2, 4), referred to as the lateral cortical network in mice  [9]. For instance, state #3 displays co-activation in the anterior and posterior cingulate areas (ACA, PCA. respectively), and retro-hippocampal formation (rHPF), characteristic of the default-mode network (Fig. 1a ). To account for differences between awake and anesthesia contributions, we reproduced the brain states in a dataset obtained with awake mice (Fig.