pwc-archive/papers-with-abstracts huggingface dataset:
Unveiling Secrets of Brain Function With Generative Modeling: Motion Perception in Primates & Cortical Network Organization in Mice
Authors: Hadi Vafaii
Date: 2024-12-25 00:00:00

This Dissertation is comprised of two main projects, addressing questions in neuroscience through applications of generative modeling. Project #1 (Chapter 4) explores how neurons encode features of the external world. I combine Helmholtz's "Perception as Unconscious Inference" -- paralleled by modern generative models like variational autoencoders (VAE) -- with the hierarchical structure of the visual cortex. This combination leads to the development of a hierarchical VAE model, which I test for its ability to mimic neurons from the primate visual cortex in response to motion stimuli. Results show that the hierarchical VAE perceives motion similar to the primate brain. Additionally, the model identifies causal factors of retinal motion inputs, such as object- and self-motion, in a completely unsupervised manner. Collectively, these results suggest that hierarchical inference underlines the brain's understanding of the world, and hierarchical VAEs can effectively model this understanding. Project #2 (Chapter 5) investigates the spatiotemporal structure of spontaneous brain activity and its reflection of brain states like rest. Using simultaneous fMRI and wide-field Ca2+ imaging data, this project demonstrates that the mouse cortex can be decomposed into overlapping communities, with around half of the cortical regions belonging to multiple communities. Comparisons reveal similarities and differences between networks inferred from fMRI and Ca2+ signals. The introduction (Chapter 1) is divided similarly to this abstract: sections 1.1 to 1.8 provide background information about Project #1, and sections 1.9 to 1.13 are related to Project #2. Chapter 2 includes historical background, Chapter 3 provides the necessary mathematical background, and finally, Chapter 6 contains concluding remarks and future directions.
Predicting Human Brain States with Transformer
Authors: Yifei Sun, Mariano Cabezas, JiAh Lee, Chenyu Wang, Wei zhang, Fernando Calamante, Jinglei Lv
Date: 2024-12-11 00:00:00
Tasks: Language Modelling, Music Generation

The human brain is a complex and highly dynamic system, and our current knowledge of its functional mechanism is still very limited. Fortunately, with functional magnetic resonance imaging (fMRI), we can observe blood oxygen level-dependent (BOLD) changes, reflecting neural activity, to infer brain states and dynamics. In this paper, we ask the question of whether the brain states rep-resented by the regional brain fMRI can be predicted. Due to the success of self-attention and the transformer architecture in sequential auto-regression problems (e.g., language modelling or music generation), we explore the possi-bility of the use of transformers to predict human brain resting states based on the large-scale high-quality fMRI data from the human connectome project (HCP). Current results have shown that our model can accurately predict the brain states up to 5.04s with the previous 21.6s. Furthermore, even though the prediction error accumulates for the prediction of a longer time period, the gen-erated fMRI brain states reflect the architecture of functional connectome. These promising initial results demonstrate the possibility of developing gen-erative models for fMRI data using self-attention that learns the functional or-ganization of the human brain. Our code is available at: https://github.com/syf0122/brain_state_pred.
Derivations of Animal Movement Models with Explicit Memory
Authors: Tianxu Wang, Kyunghan Choi, Hao Wang
Date: 2024-12-29 00:00:00

Highly evolved animals continuously update their knowledge of social factors, refining movement decisions based on both historical and real-time observations. Despite its significance, research on the underlying mechanisms remains limited. In this study, we explore how the use of explicit memory shapes different mathematical models across various ecological dispersal scenarios. Specifically, we investigate three memory-based dispersal scenarios: gradient-based movement, where individuals respond to environmental gradients; environment matching, which promotes uniform distribution within a population; and location-based movement, where decisions rely solely on local suitability. These scenarios correspond to diffusion advection, Fickian diffusion, and Fokker-Planck diffusion models, respectively. We focus on the derivation of these memory-based movement models using three approaches: spatial and temporal discretization, patch models in continuous time, and discrete-velocity jump process. These derivations highlight how different ways of using memory lead to distinct mathematical models. Numerical simulations reveal that the three dispersal scenarios exhibit distinct behaviors under memory-induced repulsive and attractive conditions. The diffusion advection and Fokker-Planck models display wiggle patterns and aggregation phenomena, while simulations of the Fickian diffusion model consistently stabilize to uniform constant states.
Intrinsic noise in structured replicator dynamics modelling time delays
Authors: Jacek Miekisz, Javad Mohamadichamgavi
Date: 2024-12-28 00:00:00

We construct and analyze structured replicator dynamics of the Snowdrift game. In our model, the offspring is put in juvenile compartments and then mature and join adult compartments with strategy-dependent rates. This is augmented by death rates and hence the population size is bounded. In the corresponding birth-death Markov jump process, rates of leaving juvenile compartments may be interpreted as inverses of averages of exponentially distributed time delays. We observe a novel behavior: for equal average time delays of both strategies, the frequency of cooperators in the quasi-stationary state of a stochastic dynamics is bigger than that in the corresponding stationary state of the deterministic structured replicator dynamics which is actually equal to the critical point of the original replicator equation for the Snowdrift game. In short, time delays favor cooperation in the presence of intrinsic stochastic fluctuations.
Leslie Population Models in Predator-prey and Competitive populations: theory and applications by machine learning
Authors: Pico Gilman, Steven J. Miller, Daeyoung Son, Saad Waheed, Janine Wang
Date: 2024-12-20 00:00:00

We introduce a new predator-prey model by replacing the growth and predation constant by a square matrix, and the population density as a population vector. The classical Lotka-Volterra model describes a population that either modulates or converges. Stability analysis of such models have been extensively studied by the works of Merdan (https://doi.org/10.1016/j.chaos.2007.06.062). The new model adds complexity by introducing an age group structure where the population of each age group evolves as prescribed by the Leslie matrix. The added complexity changes the behavior of the model such that the population either displays roughly an exponential growth or decay. We first provide an exact equation that describes a time evolution and use analytic techniques to obtain an approximate growth factor. We also discuss the variants of the Leslie model, i.e., the complex value predator-prey model and the competitive model. We then prove the Last Species Standing theorem that determines the dominant population in the large time limit. The recursive structure of the model denies the application of simple regression. We discuss a machine learning scheme that allows an admissible fit for the population evolution of Paramecium Aurelia and Paramecium Caudatum. Another potential avenue to simplify the computation is to use the machinery of quantum operators. We demonstrate the potential of this approach by computing the Hamiltonian of a simple Leslie system.
A Standardized Framework for Sensor Placement in Human Motion Capture and Wearable Applications
Authors: Seyed Yahya Shirazi, Julius Welzel, Sein Jeung, Lara Godbersen
Date: 2024-12-30 00:00:00

The proliferation of wearable sensors and monitoring technologies has created an urgent need for standardized sensor placement protocols. While existing standards like SENIAM address specific applications, no comprehensive framework spans different sensing modalities and applications. We present a unified sensor placement standard that ensures the reproducibility and transferability of human movement and physiological data across various systems and research domains. Our framework provides precise anatomical landmarks, coordinate systems, and placement protocols with defined precision levels, compatible with existing data-sharing standards such as the Brain Imaging Data Structure (BIDS) and Heirechciacal Event Descriptors (HED). This framework aims to enhance data quality, reproducibility, and interoperability in applications ranging from lab-based clinical biomechanics to continuous health monitoring in everyday life.
Pharmacometrics Modeling via Physics-Informed Neural Networks: Integrating Time-Variant Absorption Rates and Fractional Calculus for Enhancing Prediction Accuracy
Authors: Nazanin Ahmadi, Shupeng Wang, George Karniadakis
Date: 2024-12-30 00:00:00

We present a novel method to improve pharmacokinetics modeling, an essential step of drug development. Conventional models frequently fail to fully represent the intricacies of drug absorption and distribution, which limits their predictive abilities required for personalized treatment strategies. Our methodology introduces two innovations to enhance modeling accuracy: 1. Time-varying parameters: this approach is designed to accommodate the dynamic nature of drug absorption rates. 2. Fractional calculus in representing delayed drug response. This approach effectively captures anomalous diffusion phenomena, surpassing traditional models in describing drug delayed response without the need for extensive compartmentalization.
Considering experimental frame rates and robust segmentation analysis of piecewise-linear microparticle trajectories
Authors: Keisha J. Cook, Nathan Rayens, Linh Do, Christine K. Payne, Scott A. McKinley
Date: 2024-12-30 00:00:00

The movement of intracellular cargo transported by molecular motors is commonly marked by switches between directed motion and stationary pauses. The predominant measure for assessing movement is effective diffusivity, which predicts the mean-squared displacement of particles over long time scales. In this work, we consider an alternative analysis regime that focuses on shorter time scales and relies on automated segmentation of paths. Due to intrinsic uncertainty in changepoint analysis, we highlight the importance of statistical summaries that are robust with respect to the performance of segmentation algorithms. In contrast to effective diffusivity, which averages over multiple behaviors, we emphasize tools that highlight the different motor-cargo states, with an eye toward identifying biophysical mechanisms that determine emergent whole-cell transport properties. By developing a Markov chain model for noisy, continuous, piecewise-linear microparticle movement, and associated mathematical analysis, we provide insight into a common question posed by experimentalists: how does the choice of observational frame rate affect what is inferred about transport properties?
A Dynamic Spillover Effect Investigation on Cryptocurrency Market Before and After Pandemic
Authors: Wenjie Lan
Date: 2024-12-28 00:00:00

This paper distinguishes between risk resonance and risk diversification relationships in the cryptocurrency market based on the newly developed asymmetric breakpoint approach, and analyzes the risk propagation mechanism among cryptocurrencies under extreme events. In addition, through the lens of node association and network structure, this paper explores the dynamic evolutionary relationship of cryptocurrency risk association before and after the epidemic. In addition, the driving mechanism of the cryptocurrency risk movement is analyzed in a depth with the epidemic indicators. The findings show that the effect of propagation of risk among cryptocurrencies becomes more significant under the influence of the new crown outbreak. At the same time, the increase in the number of confirmed cases exacerbated the risk spillover effect among cryptocurrencies, while the risk resonance effect that exists between the crude oil market and the cryptocurrency market amplified the extent of the outbreak's impact on cryptocurrencies. However, other financial markets are relatively independent of the cryptocurrency market. This study proposes a strategy to deal with the spread of cryptocurrency risks from the perspective of a public health crisis, providing a useful reference basis for improving the regulatory mechanism of cryptocurrencies.
Strategic Learning and Trading in Broker-Mediated Markets
Authors: Alif Aqsha, Fayçal Drissi, Leandro Sánchez-Betancourt
Date: 2024-12-30 00:00:00

We study strategic interactions in a broker-mediated market. A broker provides liquidity to an informed trader and to noise traders while managing inventory in the lit market. The broker and the informed trader maximise their trading performance while filtering each other's private information; the trader estimates the broker's trading activity in the lit market while the broker estimates the informed trader's private signal. Brokers hold a strategic advantage over traders who rely solely on prices to filter information. We find that information leakage in the client's trading flow yields an economic value to the broker that is comparable to transaction costs; she speculates profitably and mitigates risk effectively, which, in turn, adversely impacts the informed trader's performance. In contrast, low signal-to-noise sources, such as prices, result in the broker's trading performance being indistinguishable from that of a naive strategy that internalises noise flow, externalises informed flow, and offloads inventory at a constant rate.
GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection
Authors: Xutao Liao, Shaohui Li, Yuhui Xu, Zhi Li, Yu Liu, You He
Date: 2024-12-15 00:00:00
Tasks: Arithmetic Reasoning, Text Generation

Recent low-rank training methods, such as GaLore, have significantly reduced the memory required to optimize large language models (LLMs). However, these methods often suffer from time-consuming low-rank projection estimations. In particular, the singular value decomposition (SVD) in GaLore can consume more than 80\% of the total training time. To address this issue, we propose GaLore$+$, which uses cross-head low-rank projection to reduce the substantial time consumption in estimating low-rank projections for multi-head attention. In addition, we employ randomized subspace iteration to achieve fast SVD. To further enhance performance, we propose sparsely coded residuals to reduce the errors caused by low-rank approximation on the first- and second-order moments of the optimizers and weight updates. We evaluate GaLore$+$ on arithmetic reasoning and natural language generation datasets. Our experiments demonstrate that GaLore$+$ delivers superior performance while achieving approximately $4\times$ fine-tuning speed compared to vanilla GaLore.
Back To The Future: A Hybrid Transformer-XGBoost Model for Action-oriented Future-proofing Nowcasting
Authors: Ziheng Sun
Date: 2024-12-21 00:00:00
Tasks: Future prediction

Inspired by the iconic movie Back to the Future, this paper explores an innovative adaptive nowcasting approach that reimagines the relationship between present actions and future outcomes. In the movie, characters travel through time to manipulate past events, aiming to create a better future. Analogously, our framework employs predictive insights about the future to inform and adjust present conditions. This dual-stage model integrates the forecasting power of Transformers (future visionary) with the interpretability and efficiency of XGBoost (decision maker), enabling a seamless loop of future prediction and present adaptation. Through experimentation with meteorological datasets, we demonstrate the framework's advantage in achieving more accurate forecasting while guiding actionable interventions for real-time applications.
A Review of Latent Representation Models in Neuroimaging
Authors: C. Vázquez-García, F. J. Martínez-Murcia, F. Segovia Román, Juan M. Górriz
Date: 2024-12-24 00:00:00

Neuroimaging data, particularly from techniques like MRI or PET, offer rich but complex information about brain structure and activity. To manage this complexity, latent representation models - such as Autoencoders, Generative Adversarial Networks (GANs), and Latent Diffusion Models (LDMs) - are increasingly applied. These models are designed to reduce high-dimensional neuroimaging data to lower-dimensional latent spaces, where key patterns and variations related to brain function can be identified. By modeling these latent spaces, researchers hope to gain insights into the biology and function of the brain, including how its structure changes with age or disease, or how it encodes sensory information, predicts and adapts to new inputs. This review discusses how these models are used for clinical applications, like disease diagnosis and progression monitoring, but also for exploring fundamental brain mechanisms such as active inference and predictive coding. These approaches provide a powerful tool for both understanding and simulating the brain's complex computational tasks, potentially advancing our knowledge of cognition, perception, and neural disorders.
Symbolic Disentangled Representations for Images
Authors: Alexandr Korchemnyi, Alexey K. Kovalev, Aleksandr I. Panov
Date: 2024-12-25 00:00:00
Tasks: Disentanglement, Object

The idea of disentangled representations is to reduce the data to a set of generative factors that produce it. Typically, such representations are vectors in latent space, where each coordinate corresponds to one of the generative factors. The object can then be modified by changing the value of a particular coordinate, but it is necessary to determine which coordinate corresponds to the desired generative factor -- a difficult task if the vector representation has a high dimension. In this article, we propose ArSyD (Architecture for Symbolic Disentanglement), which represents each generative factor as a vector of the same dimension as the resulting representation. In ArSyD, the object representation is obtained as a superposition of the generative factor vector representations. We call such a representation a \textit{symbolic disentangled representation}. We use the principles of Hyperdimensional Computing (also known as Vector Symbolic Architectures), where symbols are represented as hypervectors, allowing vector operations on them. Disentanglement is achieved by construction, no additional assumptions about the underlying distributions are made during training, and the model is only trained to reconstruct images in a weakly supervised manner. We study ArSyD on the dSprites and CLEVR datasets and provide a comprehensive analysis of the learned symbolic disentangled representations. We also propose new disentanglement metrics that allow comparison of methods using latent representations of different dimensions. ArSyD allows to edit the object properties in a controlled and interpretable way, and the dimensionality of the object property representation coincides with the dimensionality of the object representation itself.
Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing
Authors: Hao Fei, Shengqiong Wu, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan
Date: 2024-10-08 00:00:00

Recent developments of vision large language models (LLMs) have seen remarkable progress, yet still encounter challenges towards multimodal generalists, such as coarse-grained instance-level understanding, lack of unified support for both images and videos, and insufficient coverage across various vision tasks. In this paper, we present VITRON, a universal pixel-level vision LLM designed for comprehensive understanding, generating, segmenting, and editing of both static images and dynamic videos. Building on top of an LLM backbone, VITRON incorporates encoders for images, videos, and pixel-level regional visuals within its frontend modules, while employing state-of-the-art visual specialists as its backend, via which VITRON supports a spectrum of vision end tasks, spanning visual comprehension to visual generation, from low level to high level. To ensure an effective and precise message passing from LLM to backend modules for function invocation, we propose a novel hybrid method by simultaneously integrating discrete textual instructions and continuous signal embeddings. Further, we design various pixel-level spatiotemporal vision-language alignment learning for VITRON to reach the best fine-grained visual capability. Finally, a cross-task synergy module is advised to learn to maximize the task-invariant fine-grained visual features, enhancing the synergy between different visual tasks. Demonstrated over 12 visual tasks and evaluated across 22 datasets, VITRON showcases its extensive capabilities in the four main vision task clusters. Overall, this work illuminates the great potential of developing a more unified multimodal generalist. Project homepage: https://vitron-llm.github.io/
Multi-atlas Ensemble Graph Neural Network Model For Major Depressive Disorder Detection Using Functional MRI Data
Authors: Nojod M. Alotaibi, Areej M. Alhothali, Manar S. Ali
Date: 2024-12-21 00:00:00
Tasks: Diagnostic, Graph Neural Network, Specificity

Major depressive disorder (MDD) is one of the most common mental disorders, with significant impacts on many daily activities and quality of life. It stands as one of the most common mental disorders globally and ranks as the second leading cause of disability. The current diagnostic approach for MDD primarily relies on clinical observations and patient-reported symptoms, overlooking the diverse underlying causes and pathophysiological factors contributing to depression. Therefore, scientific researchers and clinicians must gain a deeper understanding of the pathophysiological mechanisms involved in MDD. There is growing evidence in neuroscience that depression is a brain network disorder, and the use of neuroimaging, such as magnetic resonance imaging (MRI), plays a significant role in identifying and treating MDD. Rest-state functional MRI (rs-fMRI) is among the most popular neuroimaging techniques used to study MDD. Deep learning techniques have been widely applied to neuroimaging data to help with early mental health disorder detection. Recent years have seen a rise in interest in graph neural networks (GNNs), which are deep neural architectures specifically designed to handle graph-structured data like rs-fMRI. This research aimed to develop an ensemble-based GNN model capable of detecting discriminative features from rs-fMRI images for the purpose of diagnosing MDD. Specifically, we constructed an ensemble model by combining features from multiple brain region segmentation atlases to capture brain complexity and detect distinct features more accurately than single atlas-based models. Further, the effectiveness of our model is demonstrated by assessing its performance on a large multi-site MDD dataset. The best performing model among all folds achieved an accuracy of 75.80%, a sensitivity of 88.89%, a specificity of 61.84%, a precision of 71.29%, and an F1-score of 79.12%.