pwc-archive/papers-with-abstracts huggingface dataset:
PepDoRA: A Unified Peptide Language Model via Weight-Decomposed Low-Rank Adaptation
Authors: Leyao Wang, Rishab Pulugurta, Pranay Vure, Yinuo Zhang, Aastha Pal, Pranam Chatterjee
Date: 2024-10-28 00:00:00
Tasks: Activity Prediction, Contrastive Learning, Language Modeling, Language Modelling, Property Prediction

Peptide therapeutics, including macrocycles, peptide inhibitors, and bioactive linear peptides, play a crucial role in therapeutic development due to their unique physicochemical properties. However, predicting these properties remains challenging. While structure-based models primarily focus on local interactions, language models are capable of capturing global therapeutic properties of both modified and linear peptides. Protein language models like ESM-2, though effective for natural peptides, cannot however encode chemical modifications. Conversely, pre-trained chemical language models excel in representing small molecule properties but are not optimized for peptides. To bridge this gap, we introduce PepDoRA, a unified peptide representation model. Leveraging Weight-Decomposed Low-Rank Adaptation (DoRA), PepDoRA efficiently fine-tunes the ChemBERTa-77M-MLM on a masked language model objective to generate optimized embeddings for downstream property prediction tasks involving both modified and unmodified peptides. By tuning on a diverse and experimentally valid set of 100,000 modified, bioactive, and binding peptides, we show that PepDoRA embeddings capture functional properties of input peptides, enabling the accurate prediction of membrane permeability, non-fouling and hemolysis propensity, and via contrastive learning, target protein-specific binding. Overall, by providing a unified representation for chemically and biologically diverse peptides, PepDoRA serves as a versatile tool for function and activity prediction, facilitating the development of peptide therapeutics across a broad spectrum of applications.
GATES: Graph Attention Network with Global Expression Fusion for Deciphering Spatial Transcriptome Architectures
Authors: Xiongtao Xiao, Xiaofeng Chen, Feiyan Jiang, Songming Zhang, Wenming Cao, Cheng Tan, Zhangyang Gao, Zhongshan Li
Date: 2024-10-26 00:00:00
Tasks: Graph Attention

Single-cell spatial transcriptomics (ST) offers a unique approach to measuring gene expression profiles and spatial cell locations simultaneously. However, most existing ST methods assume that cells in closer spatial proximity exhibit more similar gene expression patterns. Such assumption typically results in graph structures that prioritize local spatial information while overlooking global patterns, limiting the ability to fully capture the broader structural features of biological tissues. To overcome this limitation, we propose GATES (Graph Attention neTwork with global Expression fuSion), a novel model designed to capture structural details in spatial transcriptomics data. GATES first constructs an expression graph that integrates local and global information by leveraging both spatial proximity and gene expression similarity. The model then employs an autoencoder with adaptive attention to assign proper weights for neighboring nodes, enhancing its capability of feature extraction. By fusing features of both the spatial and expression graphs, GATES effectively balances spatial context with gene expression data. Experimental results across multiple datasets demonstrate that GATES significantly outperforms existing methods in identifying spatial domains, highlighting its potential for analyzing complex biological tissues. Our code can be accessed on GitHub at https://github.com/xiaoxiongtao/GATES.
Peripheral brain interfacing: Reading high-frequency brain signals from the output of the nervous system
Authors: Jaime Ibáñez, Blanka Zicher, Etienne Burdet, Stuart N. Baker, Carsten Mehring, Dario Farina
Date: 2024-10-28 00:00:00

Accurate and robust recording and decoding from the central nervous system (CNS) is essential for advances in human-machine interfacing. However, technologies used to directly measure CNS activity are limited by their resolution, sensitivity to interferences, and invasiveness. Advances in muscle recordings and deep learning allow us to decode the spiking activity of spinal motor neurons (MNs) in real time and with high accuracy. MNs represent the motor output layer of the CNS, receiving and sampling signals originating in different regions in the nervous system, and generating the neural commands that control muscles. The input signals to MNs can be estimated from the MN outputs. Here we argue that peripheral neural interfaces using muscle sensors represent a promising, non-invasive approach to estimate some neural activity from the CNS that reaches the MNs but does not directly modulate force production. We also discuss the evidence supporting this concept, and the necessary advances to consolidate and test MN-based CNS interfaces in controlled and real-world settings.
EEG Based Decoding of the Perception and Regulation of Taboo Words
Authors: Parisa Ahmadi Ghomroudi, Michele Scaltritti, Bianca Monachesi, Peera Wongupparaj, Remo Job, Alessandro Grecucci
Date: 2024-10-25 00:00:00
Tasks: EEG, ERP

In daily interactions, emotions are frequently conveyed and triggered through verbal exchanges. Sometimes, we must modulate our emotional reactions to align with societal norms. Among the emotional words, taboo words represent a specific category that has been poorly studied. One intriguing question is whether these word categories can be predicted from EEG responses with the use of machine learning methods. To address this question, Support Vector Machine (SVM) was applied to decode the word categories from Event Related Potential (ERP) in 40 native Italian speakers. 240 neutral, negative and taboo words were used to this aim. Results indicate that the SVM classifier successfully distinguished between the three-word categories, with significant differences in neural activity ascribed to the late positive potential mainly detected in the central-parietal-occipital and anterior right scalp areas in the time windows of 450-649 ms and 650-850 ms. These findings were in line with the established distribution pattern of the late positive potential. Intriguingly, the study also revealed that word categories were still detectable in the regulate condition. This study extends previous results on the domain of the cortical responses of taboo words, and how machine learning methods can be used to predict word categories from EEG responses.
The Geometry of Concepts: Sparse Autoencoder Feature Structure
Authors: Yuxiao Li, Eric J. Michaud, David D. Baek, Joshua Engels, Xiaoqing Sun, Max Tegmark
Date: 2024-10-10 00:00:00
Tasks: Math

Sparse autoencoders have recently produced dictionaries of high-dimensional vectors corresponding to the universe of concepts represented by large language models. We find that this concept universe has interesting structure at three levels: 1) The "atomic" small-scale structure contains "crystals" whose faces are parallelograms or trapezoids, generalizing well-known examples such as (man-woman-king-queen). We find that the quality of such parallelograms and associated function vectors improves greatly when projecting out global distractor directions such as word length, which is efficiently done with linear discriminant analysis. 2) The "brain" intermediate-scale structure has significant spatial modularity; for example, math and code features form a "lobe" akin to functional lobes seen in neural fMRI images. We quantify the spatial locality of these lobes with multiple metrics and find that clusters of co-occurring features, at coarse enough scale, also cluster together spatially far more than one would expect if feature geometry were random. 3) The "galaxy" scale large-scale structure of the feature point cloud is not isotropic, but instead has a power law of eigenvalues with steepest slope in middle layers. We also quantify how the clustering entropy depends on the layer.
Linton Stereo Illusion: Response on Johnston (1991)
Authors: Paul Linton
Date: 2024-10-04 00:00:00

In (Linton, 2024) I present a new illusion (the 'Linton Stereo Illusion') that challenges our understanding of stereo vision. A vision scientist has shared their own analysis of the 'Linton Stereo Illusion' (titled: 'There is no challenge to our understanding of stereo vision: Response to Linton and Kriegeskorte (ECVP 2024 and ArXiv:2408.00770)') claiming that the 'Linton Stereo Illusion' is fully explained by Johnston (1991). I regard Johnston (1991) as one of the most important stereo vision papers in our young (< 200-year-old) field, and so this challenge requires a response. In this paper I explain why Johnston (1991) cannot explain the 'Linton Stereo Illusion'. Indeed, Johnston (1991) makes predictions that are the exact opposite of those observed in the 'Linton Stereo Illusion'. I also highlight a key concern with Johnston (1991)'s account that has so far been overlooked. Johnston (1991)'s account predicts that vergence eye movements will cause massive stereo distortions, leading to a world of unstable stereo perception. But this simply does not reflect our visual experience.
The Effect of Acute Stress on the Interpretability and Generalization of Schizophrenia Predictive Machine Learning Models
Authors: Gideon Vos, Maryam Ebrahimpour, Liza van Eijk, Zoltan Sarnyai, Mostafa Rahimi Azghadi
Date: 2024-10-04 00:00:00
Tasks: EEG

Introduction Schizophrenia is a severe mental disorder, and early diagnosis is key to improving outcomes. Its complexity makes predicting onset and progression challenging. EEG has emerged as a valuable tool for studying schizophrenia, with machine learning increasingly applied for diagnosis. This paper assesses the accuracy of ML models for predicting schizophrenia and examines the impact of stress during EEG recording on model performance. We integrate acute stress prediction into the analysis, showing that overlapping conditions like stress during recording can negatively affect model accuracy. Methods Four XGBoost models were built: one for stress prediction, two to classify schizophrenia (at rest and task), and a model to predict schizophrenia for both conditions. XAI techniques were applied to analyze results. Experiments tested the generalization of schizophrenia models using their datasets' healthy controls and independent health-screened controls. The stress model identified high-stress subjects, who were excluded from further analysis. A novel method was used to adjust EEG frequency band power to remove stress artifacts, improving predictive model performance. Results Our results show that acute stress vary across EEG sessions, affecting model performance and accuracy. Generalization improved once these varying stress levels were considered and compensated for during model training. Our findings highlight the importance of thorough health screening and management of the patient's condition during the process. Stress induced during or by the EEG recording can adversely affect model generalization. This may require further preprocessing of data by treating stress as an additional physiological artifact. Our proposed approach to compensate for stress artifacts in EEG data used for training models showed a significant improvement in predictive performance.
Multiple power laws and scaling relation in exploratory locomotion of the snail Tegula nigerrima
Authors: Katsushi Kagaya, Tomoyuki Nakano, Ryo Nakayama
Date: 2024-10-28 00:00:00
Tasks: Relation

One of goals in soft robotics is to achive spontaneous behavior like real organisms. To gain a clue to achieve this, we examined the long (16-hour) spontaneous exploratory locomotion of snails. The active forager snail, Tegula nigerrima, from an intertidal rocky shore was selected to test the general hypothesis that nervous systems are inherently near a critical state, which is self-organized to drive spontaneous animal behavior. This hypothesis, known as the critical brain hypothesis, was originally proposed for vertebrate species, but it might be applicable to other invertebrate species as well. We first investigated the power spectra of the speed of locomotion of the snails ($N=39$). The spectra showed $1/{f^\alpha}$ fluctuation, which is one of the signatures of self-organized criticality. The $\alpha$ was estimated to be about 0.9. We further examined whether the spatial and temporal quantities show multiple power-laws and scaling relations, which are rigorous criteria of criticality. Although the satisfaction of these criteria is limited to a truncated region and provides limited evidence to demonstrate the aspect of self-organization, the multiple power-laws and the scaling relations were overall satisfied. Therefore, these results additionally support the generality of the critical brain hypothesis.
The final size and critical times of an SIVR epidemic model
Authors: Phyu Phyu Win, Zhigui Lin, Mengyun Zhang
Date: 2024-10-27 00:00:00

To understand the impact of vaccination, we consider a SIVR (susceptible-infected-vaccinated-recovered) model which combines impulsive vaccination into the classical SIR model. The final size is firstly defined and estimated, and then the peak value and peak time are considered. Finally, the critical times for a given infected number is studied, and it can be used to define and estimate the stopping time. Our results extend those for the well-understood SIR model.
Neural Network-Enhanced Disease Spread Dynamics Over Time and Space
Authors: Randy L. Caga-anan
Date: 2024-10-27 00:00:00
Tasks: Diversity

This study presents a neural network-enhanced approach to modeling disease spread dynamics over time and space. Neural networks are used to estimate time-varying parameters, with two calibration methods explored: Approximate Bayesian Computation (ABC) with Trust Region Reflective (TRF) optimization, and backpropagation with the Adam optimizer. Simulations show that the second method is faster for larger networks, while the first offers a greater diversity of acceptable solutions. The model is extended spatially by introducing a pathogen compartment, which diffuses through environmental transmission and interpersonal contact. We examine scenarios of exact reporting, overreporting, and underreporting, highlighting their effects on public behavior and infection peaks. Our results demonstrate that neural network-enhanced models more accurately capture dynamic changes in disease spread and illustrate the potential of scientific machine learning (SciML) to improve the predictive power of epidemiological models beyond traditional approaches.
ZIF-90 treats fungal keratitis by promoting macrophage apoptosis and inhibiting inflammatory response
Authors: Xueyun Fu, Jing Lin, Qian Wang, Lina Zhang, Ziyi Wang, Menghui Chi, Daohao Li, Guiqiu Zhao, Cui Li
Date: 2024-10-28 00:00:00
Tasks: Prognosis

Fungal keratitis is a severe vision-threatening corneal infection with a prognosis influenced by fungal virulence and the host's immune defense mechanisms. The immune system, through its regulation of the inflammatory response, ensures cells and tissues can effectively activate defense mechanisms in response to infection and injury. However, there is still a lack of effective drugs that attenuate fungal virulence while relieving the inflammatory response caused by fungal keratitis. Therefore, finding effective treatments to solve these problems is particularly important. We synthesized ZIF-90 by water-based synthesis and characterized by SEM, XRD etc. In vitro experiments included CCK-8 and ELISA. These evaluations verified the disruptive effects of ZIF-90 on Aspergillus. fumigatus spore adhesion, morphology, cell membrane, and the effect of ZIF-90 on apoptosis. In addition, to investigate whether the metal-ligand zinc and the organic ligand imidazole act as essential factors in ZIF-90, we investigated the in vitro antimicrobial and anti-inflammatory effects of ZIF-8, ZIF-67, and MOF-74 (Zn) by MIC and ELISA experiments. ZIF-90 has therapeutic effects on fungal keratitis, which could break the protective organelles of Aspergillus. fumigatus, such as the cell wall. In addition, ZIF-90 can avoid excessive inflammatory response by promoting apoptosis of inflammatory cells. The results demonstrated that both zinc ions and imidazole possessed antimicrobial and anti-inflammatory effects. In addition, ZIF-90 exhibited better biocompatibility compared to ZIF-8, ZIF-67, and MOF-74 (Zn). ZIF-90 has anti-inflammatory and antifungal effects and preferable biocompatibility, and has great potential for the treatment of fungal keratitis.
Modeling and Replication of the Prepayment Option of Mortgages including Behavioral Uncertainty
Authors: Leonardo Perotti, Lech A. Grzelak, Cornelis W. Oosterlee
Date: 2024-10-28 00:00:00

Prepayment risk embedded in fixed-rate mortgages forms a significant fraction of a financial institution's exposure, and it receives particular attention because of the magnitude of the underlying market. The embedded prepayment option (EPO) bears the same interest rate risk as an exotic interest rate swap (IRS) with a suitable stochastic notional. We investigate the effect of relaxing the assumption of a deterministic relationship between the market interest rate incentive and the prepayment rate. A non-hedgeable risk factor is modeled to capture the uncertainty in mortgage owners' behavior, leading to an incomplete market. We prove under natural assumptions that including behavioral uncertainty reduces the exposure's value. We statically replicate the exposure resulting from the EPO with IRSs and swaptions, and we show that a replication based on swaps solely cannot easily control the right tail of the exposure distribution, while including swaptions enables that. The replication framework is flexible and focuses on different regions in the exposure distribution. Since a non-hedgeable risk factor entails the existence of multiple equivalent martingale measures, pricing and optimal replication are not unique. We investigate the effect of a market price of risk misspecification and we provide a methodology to generate robust hedging strategies. Such strategies, obtained as solutions to a saddle-point problem, allow us to bound the exposure against a misspecification of the pricing measure.
Extracting Alpha from Financial Analyst Networks
Authors: Dragos Gorduza, Yaxuan Kong, Xiaowen Dong, Stefan Zohren
Date: 2024-10-27 00:00:00
Tasks: Graph Attention

We investigate the effectiveness of a momentum trading signal based on the coverage network of financial analysts. This signal builds on the key information-brokerage role financial sell-side analysts play in modern stock markets. The baskets of stocks covered by each analyst can be used to construct a network between firms whose edge weights represent the number of analysts jointly covering both firms. Although the link between financial analysts coverage and co-movement of firms' stock prices has been investigated in the literature, little effort has been made to systematically learn the most effective combination of signals from firms covered jointly by analysts in order to benefit from any spillover effect. To fill this gap, we build a trading strategy which leverages the analyst coverage network using a graph attention network. More specifically, our model learns to aggregate information from individual firm features and signals from neighbouring firms in a node-level forecasting task. We develop a portfolio based on those predictions which we demonstrate to exhibit an annualized returns of 29.44% and a Sharpe ratio of 4.06 substantially outperforming market baselines and existing graph machine learning based frameworks. We further investigate the performance and robustness of this strategy through extensive empirical analysis. Our paper represents one of the first attempts in using graph machine learning to extract actionable knowledge from the analyst coverage network for practical financial applications.
MCI-GRU: Stock Prediction Model Based on Multi-Head Cross-Attention and Improved GRU
Authors: Peng Zhu, Yuante Li, Yifan Hu, Sheng Xiang, Qinyuan Liu, Dawei Cheng, Yuqi Liang
Date: 2024-09-25 00:00:00
Tasks: Prediction, Stock Prediction

As financial markets grow increasingly complex in the big data era, accurate stock prediction has become more critical. Traditional time series models, such as GRUs, have been widely used but often struggle to capture the intricate nonlinear dynamics of markets, particularly in the flexible selection and effective utilization of key historical information. Recently, methods like Graph Neural Networks and Reinforcement Learning have shown promise in stock prediction but require high data quality and quantity, and they tend to exhibit instability when dealing with data sparsity and noise. Moreover, the training and inference processes for these models are typically complex and computationally expensive, limiting their broad deployment in practical applications. Existing approaches also generally struggle to capture unobservable latent market states effectively, such as market sentiment and expectations, microstructural factors, and participant behavior patterns, leading to an inadequate understanding of market dynamics and subsequently impact prediction accuracy. To address these challenges, this paper proposes a stock prediction model, MCI-GRU, based on a multi-head cross-attention mechanism and an improved GRU. First, we enhance the GRU model by replacing the reset gate with an attention mechanism, thereby increasing the model's flexibility in selecting and utilizing historical information. Second, we design a multi-head cross-attention mechanism for learning unobservable latent market state representations, which are further enriched through interactions with both temporal features and cross-sectional features. Finally, extensive experiments on four main stock markets show that the proposed method outperforms SOTA techniques across multiple metrics. Additionally, its successful application in real-world fund management operations confirms its effectiveness and practicality.
Optimal life insurance and annuity decision under money illusion
Authors: Wenyuan Li, Pengyu Wei
Date: 2024-10-26 00:00:00

This paper investigates the optimal consumption, investment, and life insurance/annuity decisions for a family in an inflationary economy under money illusion. The family can invest in a financial market that consists of nominal bonds, inflation-linked bonds, and a stock index. The breadwinner can also purchase life insurance or annuities that are available continuously. The family's objective is to maximize the expected utility of a mixture of nominal and real consumption, as they partially overlook inflation and tend to think in terms of nominal rather than real monetary values. We formulate this life-cycle problem as a random horizon utility maximization problem and derive the optimal strategy. We calibrate our model to the U.S. data and demonstrate that money illusion increases life insurance demand for young adults and reduces annuity demand for retirees. Our findings indicate that the money illusion contributes to the annuity puzzle and highlights the role of financial literacy in an inflationary environment.