pwc-archive/papers-with-abstracts huggingface dataset:
Leveraging Deep Generative Model For Computational Protein Design And Optimization
Authors: Boqiao Lai
Date: 2024-08-30 00:00:00
Tasks: Protein Design, Protein Structure Prediction

Proteins are the fundamental macromolecules that play diverse and crucial roles in all living matter and have tremendous implications in healthcare, manufacturing, and biotechnology. Their functions are largely determined by the sequences of amino acids that compose them and their unique three-dimensional structures when folded. The recent surge in highly accurate computational protein structure prediction tools has equipped scientists with the means to derive preliminary structural insights without the onerous costs of experimental structure determination. These breakthroughs hold profound promise for building robust and efficient in silico protein design systems. While the prospect of designing de novo proteins with precise computational accuracy remains a grand challenge in biochemical engineering, conventional assembly-based and rational design methods often grapple with the expansive design space, resulting in suboptimal design success rates. Despite recently emerged deep learning-based models have shown promise in improving the efficiency of the computational protein design process, a significant gap persists between current design paradigms and their experimental realization. This thesis will investigate the potential of deep generative models in refining protein structure and sequence design methods, aiming to develop frameworks capable of crafting novel protein sequences with predetermined structures or specific functionalities. By harnessing extensive protein databases and cutting-edge neural architectures, this research aims to enhance precision and robustness in current protein design paradigms, potentially paving the way for advancements across various scientific fields.
Multi-faceted Neuroimaging Data Integration via Analysis of Subspaces
Authors: Andrew Ackerman, Zhengwu Zhang, Jan Hannig, Jack Prothero, J. S. Marron
Date: 2024-08-28 00:00:00
Tasks: Data Integration, Functional Connectivity

Neuroimaging studies, such as the Human Connectome Project (HCP), often collect multi-faceted and multi-block data to study the complex human brain. However, these data are often analyzed in a pairwise fashion, which can hinder our understanding of how different brain-related measures interact with each other. In this study, we comprehensively analyze the multi-block HCP data using the Data Integration via Analysis of Subspaces (DIVAS) method. We integrate structural and functional brain connectivity, substance use, cognition, and genetics in an exhaustive five-block analysis. This gives rise to the important finding that genetics is the single data modality most predictive of brain connectivity, outside of brain connectivity itself. Nearly 14\% of the variation in functional connectivity (FC) and roughly 12\% of the variation in structural connectivity (SC) is attributed to shared spaces with genetics. Moreover, investigations of shared space loadings provide interpretable associations between particular brain regions and drivers of variability, such as alcohol consumption in the substance-use data block. Novel Jackstraw hypothesis tests are developed for the DIVAS framework to establish statistically significant loadings. For example, in the (FC, SC, and Substance Use) shared space, these novel hypothesis tests highlight largely negative functional and structural connections suggesting the brain's role in physiological responses to increased substance use. Furthermore, our findings have been validated using a subset of genetically relevant siblings or twins not studied in the main analysis.
Ultra-rapid, Quantitative, Label-free Antibiotic Susceptibility Testing via Optically Detected Purine Metabolites
Authors: Allen Fraiman, Lawrence Ziegler
Date: 2024-08-30 00:00:00

There is an urgent need for the development of novel and truly rapid (equal or less than 1 hour) antibiotic susceptibility testing (AST) platforms in order to provide best antimicrobial prescribing practices and to help reduce the increasing global threat of antibiotic resistance. A 785 nm surface enhanced Raman spectroscopy (SERS) based phenotypic methodology is described that results in accurate minimum inhibitory concentration (MIC) determinations for all tested strain-antibiotic pairs. The SERS-AST procedure results in accurate MICs, the key quantitative measure of in vitro drug susceptibility, in 1 hour, including a 30-minute incubation period. The method is effective for both Gram positive and negative species, and for antibiotics with different initial primary targets of antibiotic activity, and for both bactericidal and bacteriostatic antibiotics. The molecular level mechanism of this methodology is described. Bacterial SERS spectra are due to secreted purine nucleotide degradation products (principally adenine, guanine, xanthine and hypoxanthine) resulting from water washing induced bacterial stringent response and the resulting (p)ppGpp alarmone mediates nucleobase formation from unneeded tRNA and rRNA. The rewiring of metabolic responses resulting from the secondary metabolic effects of antibiotic exposure during the 30-minute incubation period accounts for the dose dependence of the SERS spectral intensities which are used to accurately yield the MIC. This is the fastest demonstrated AST method yielding MICs.
An Effective Information Theoretic Framework for Channel Pruning
Authors: Yihao Chen, Zefang Wang
Date: 2024-08-14 00:00:00
Tasks: Model Compression

Channel pruning is a promising method for accelerating and compressing convolutional neural networks. However, current pruning algorithms still remain unsolved problems that how to assign layer-wise pruning ratios properly and discard the least important channels with a convincing criterion. In this paper, we present a novel channel pruning approach via information theory and interpretability of neural networks. Specifically, we regard information entropy as the expected amount of information for convolutional layers. In addition, if we suppose a matrix as a system of linear equations, a higher-rank matrix represents there exist more solutions to it, which indicates more uncertainty. From the point of view of information theory, the rank can also describe the amount of information. In a neural network, considering the rank and entropy as two information indicators of convolutional layers, we propose a fusion function to reach a compromise of them, where the fusion results are defined as ``information concentration''. When pre-defining layer-wise pruning ratios, we employ the information concentration as a reference instead of heuristic and engineering tuning to provide a more interpretable solution. Moreover, we leverage Shapley values, which are a potent tool in the interpretability of neural networks, to evaluate the channel contributions and discard the least important channels for model compression while maintaining its performance. Extensive experiments demonstrate the effectiveness and promising performance of our method. For example, our method improves the accuracy by 0.21% when reducing 45.5% FLOPs and removing 40.3% parameters for ResNet-56 on CIFAR-10. Moreover, our method obtains loss in Top-1/Top-5 accuracies of 0.43%/0.11% by reducing 41.6% FLOPs and removing 35.0% parameters for ResNet-50 on ImageNet.
Online Behavior Modification for Expressive User Control of RL-Trained Robots
Authors: Isaac Sheidlower, Mavis Murdock, Emma Bethel, Reuben M. Aronson, Elaine Schaertl Short
Date: 2024-08-15 00:00:00
Tasks: Diversity, Reinforcement Learning (RL)

Reinforcement Learning (RL) is an effective method for robots to learn tasks. However, in typical RL, end-users have little to no control over how the robot does the task after the robot has been deployed. To address this, we introduce the idea of online behavior modification, a paradigm in which users have control over behavior features of a robot in real time as it autonomously completes a task using an RL-trained policy. To show the value of this user-centered formulation for human-robot interaction, we present a behavior diversity based algorithm, Adjustable Control Of RL Dynamics (ACORD), and demonstrate its applicability to online behavior modification in simulation and a user study. In the study (n=23) users adjust the style of paintings as a robot traces a shape autonomously. We compare ACORD to RL and Shared Autonomy (SA), and show ACORD affords user-preferred levels of control and expression, comparable to SA, but with the potential for autonomous execution and robustness of RL.
Evolution of binary stars and the effect of tides on binary populations
Authors: Jarrod R. Hurley, Christopher A. Tout, Onno R. Pols
Date: 2002-01-14 00:00:00
Tasks: Friction

We present a rapid binary evolution algorithm that enables modelling of even the most complex binary systems. In addition to all aspects of single star evolution, features such as mass transfer, mass accretion, common-envelope evolution, collisions, supernova kicks and angular momentum loss mechanisms are included. In particular, circularization and synchronization of the orbit by tidal interactions are calculated for convective, radiative and degenerate damping mechanisms. We use this algorithm to study the formation and evolution of various binary systems. We also investigate the effect that tidal friction has on the outcome of binary evolution. Using the rapid binary code, we generate a series of large binary populations and evaluate the formation rate of interesting individual species and events. By comparing the results for populations with and without tidal friction we quantify the hitherto ignored systematic effect of tides and show that modelling of tidal evolution in binary systems is necessary in order to draw accurate conclusions from population synthesis work. Tidal synchronism is important but because orbits generally circularize before Roche-lobe overflow the outcome of the interactions of systems with the same semi-latus rectum is almost independent of eccentricity. It is not necessary to include a distribution of eccentricities in population synthesis of interacting binaries, however, the initial separations should be distributed according to the observed distribution of semi-latera recta rather than periods or semi-major axes.
SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models
Authors: Yang Cao
Date: 2024-08-21 00:00:00
Tasks: 8k, GSM8K, Math, parameter-efficient fine-tuning

In this paper, we propose Singular Values and Orthonormal Regularized Singular Vectors Adaptation, or SORSA, a novel PEFT method. Each SORSA adapter consists of two main parts: trainable principal singular weights $W_p = U_p \text{diag}(S_p) V^\top_p$, and frozen residual weights $W_r = U_r \text{diag}(S_r) V^\top_r$. These parts are initialized by performing singular value decomposition (SVD) on pre-trained weights. Moreover, we implement and analyze an orthonormal regularizer, which we prove could decrease the condition number of $W_p$ and make the optimization more efficient. SORSA adapters could be merged during inference, thus eliminating any inference latency. We also introduce a method to analyze the variation of the parameters by performing SVD and discuss and analyze SORSA's superiority in minimizing the alteration in the SVD aspect. After all, SORSA shows a faster convergence than LoRA and PiSSA in our experiments. On the GSM-8K benchmark, Llama 2 7B adapted using SORSA achieved 56.03% accuracy, surpassing LoRA (42.30%), AdaLoRA (47.30%), Full FT (49.05%), and PiSSA (53.07%). On the MATH benchmark, SORSA achieved 10.36% accuracy, outperforming LoRA (5.50%), AdaLoRA (6.48%), Full FT (7.22%), and PiSSA (7.44%). We conclude that SORSA offers a new perspective on parameter-efficient fine-tuning, demonstrating remarkable performance.
Separation of Body and Background in Radiological Images. A Practical Python Code
Authors: Seyedeh Fahimeh Hosseini, Faezeh Shalbafzadeh, Behzad Amanpour-Gharaei
Date: 2024-08-31 00:00:00
Tasks: Computed Tomography (CT)

Radiological images, such as magnetic resonance imaging (MRI) and computed tomography (CT) images, typically consist of a body part and a dark background. For many analyses, it is necessary to separate the body part from the background. In this article, we present a Python code designed to separate body and background regions in 2D and 3D radiological images. We tested the algorithm on various MRI and CT images of different body parts, including the brain, neck, and abdominal regions. Additionally, we introduced a method for intensity normalization and outlier restriction, adjusted for data conversion into 8-bit unsigned integer (UINT8) format, and examined its effects on body-background separation. Our Python code is available for use with proper citation.
Aligning Medical Images with General Knowledge from Large Language Models
Authors: Xiao Fang, Yi Lin, Dong Zhang, Kwang-Ting Cheng, Hao Chen
Date: 2024-08-31 00:00:00
Tasks: General Knowledge, Medical Image Analysis, Prompt Learning, Representation Learning, Transfer Learning

Pre-trained large vision-language models (VLMs) like CLIP have revolutionized visual representation learning using natural language as supervisions, and demonstrated promising generalization ability. In this work, we propose ViP, a novel visual symptom-guided prompt learning framework for medical image analysis, which facilitates general knowledge transfer from CLIP. ViP consists of two key components: a visual symptom generator (VSG) and a dual-prompt network. Specifically, VSG aims to extract explicable visual symptoms from pre-trained large language models, while the dual-prompt network utilizes these visual symptoms to guide the training on two learnable prompt modules, i.e., context prompt and merge prompt, which effectively adapts our framework to medical image analysis via large VLMs. Extensive experimental results demonstrate that ViP can outperform state-of-the-art methods on two challenging datasets.
Spatially-Aware Diffusion Models with Cross-Attention for Global Field Reconstruction with Sparse Observations
Authors: Yilin Zhuang, Sibo Cheng, Karthik Duraisamy
Date: 2024-08-30 00:00:00
Tasks: Inductive Bias

Diffusion models have gained attention for their ability to represent complex distributions and incorporate uncertainty, making them ideal for robust predictions in the presence of noisy or incomplete data. In this study, we develop and enhance score-based diffusion models in field reconstruction tasks, where the goal is to estimate complete spatial fields from partial observations. We introduce a condition encoding approach to construct a tractable mapping mapping between observed and unobserved regions using a learnable integration of sparse observations and interpolated fields as an inductive bias. With refined sensing representations and an unraveled temporal dimension, our method can handle arbitrary moving sensors and effectively reconstruct fields. Furthermore, we conduct a comprehensive benchmark of our approach against a deterministic interpolation-based method across various static and time-dependent PDEs. Our study attempts to addresses the gap in strong baselines for evaluating performance across varying sampling hyperparameters, noise levels, and conditioning methods. Our results show that diffusion models with cross-attention and the proposed conditional encoding generally outperform other methods under noisy conditions, although the deterministic method excels with noiseless data. Additionally, both the diffusion models and the deterministic method surpass the numerical approach in accuracy and computational cost for the steady problem. We also demonstrate the ability of the model to capture possible reconstructions and improve the accuracy of fused results in covariance-based correction tasks using ensemble sampling.
Learning linear acyclic causal model including Gaussian noise using ancestral relationships
Authors: Ming Cai, Penggang Gao, Hisayuki Hara
Date: 2024-08-31 00:00:00

This paper discusses algorithms for learning causal DAGs. The PC algorithm makes no assumptions other than the faithfulness to the causal model and can identify only up to the Markov equivalence class. LiNGAM assumes linearity and continuous non-Gaussian disturbances for the causal model, and the causal DAG defining LiNGAM is shown to be fully identifiable. The PC-LiNGAM, a hybrid of the PC algorithm and LiNGAM, can identify up to the distribution-equivalence pattern of a linear causal model, even in the presence of Gaussian disturbances. However, in the worst case, the PC-LiNGAM has factorial time complexity for the number of variables. In this paper, we propose an algorithm for learning the distribution-equivalence patterns of a linear causal model with a lower time complexity than PC-LiNGAM, using the causal ancestor finding algorithm in Maeda and Shimizu, which is generalized to account for Gaussian disturbances.
Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning
Authors: Jinglin Liang, Jin Zhong, Hanlin Gu, Zhongqi Lu, Xingxing Tang, Gang Dai, Shuangping Huang, Lixin Fan, Qiang Yang
Date: 2024-09-02 00:00:00
Tasks: Continual Learning, Contrastive Learning, Data-free Knowledge Distillation, Domain Generalization, Knowledge Distillation

Federated Class Continual Learning (FCCL) merges the challenges of distributed client learning with the need for seamless adaptation to new classes without forgetting old ones. The key challenge in FCCL is catastrophic forgetting, an issue that has been explored to some extent in Continual Learning (CL). However, due to privacy preservation requirements, some conventional methods, such as experience replay, are not directly applicable to FCCL. Existing FCCL methods mitigate forgetting by generating historical data through federated training of GANs or data-free knowledge distillation. However, these approaches often suffer from unstable training of generators or low-quality generated data, limiting their guidance for the model. To address this challenge, we propose a novel method of data replay based on diffusion models. Instead of training a diffusion model, we employ a pre-trained conditional diffusion model to reverse-engineer each class, searching the corresponding input conditions for each class within the model's input space, significantly reducing computational resources and time consumption while ensuring effective generation. Furthermore, we enhance the classifier's domain generalization ability on generated and real data through contrastive learning, indirectly improving the representational capability of generated data for real data. Comprehensive experiments demonstrate that our method significantly outperforms existing baselines. Code is available at https://github.com/jinglin-liang/DDDR.
Multi-Output Distributional Fairness via Post-Processing
Authors: Gang Li, Qihang Lin, Ayush Ghosh, Tianbao Yang
Date: 2024-08-31 00:00:00
Tasks: Fairness, Multi-class Classification, Representation Learning

The post-processing approaches are becoming prominent techniques to enhance machine learning models' fairness because of their intuitiveness, low computational cost, and excellent scalability. However, most existing post-processing methods are designed for task-specific fairness measures and are limited to single-output models. In this paper, we introduce a post-processing method for multi-output models, such as the ones used for multi-task/multi-class classification and representation learning, to enhance a model's distributional parity, a task-agnostic fairness measure. Existing methods for achieving distributional parity rely on the (inverse) cumulative density function of a model's output, restricting their applicability to single-output models. Extending previous works, we propose to employ optimal transport mappings to move a model's outputs across different groups towards their empirical Wasserstein barycenter. An approximation technique is applied to reduce the complexity of computing the exact barycenter and a kernel regression method is proposed to extend this process to out-of-sample data. Our empirical studies evaluate the proposed approach against various baselines on multi-task/multi-class classification and representation learning tasks, demonstrating the effectiveness of the proposed approach.
Learning and Control from Similarity Between Heterogeneous Systems: A Behavioral Approach
Authors: Chenchao Wang, Deyuan Meng
Date: 2024-09-01 00:00:00

This paper proposes basic definitions of similarity and similarity indexes between heterogeneous linear systems and presents a similarity-based learning control strategy. By exploring geometric properties of admissible behaviors of linear systems, the similarity indexes between two admissible behaviors of heterogeneous systems are defined as the principal angles between their subspace components, and an efficient strategy for calculating the similarity indexes is developed. By leveraging the similarity indexes, a similarity-based learning control strategy is proposed via projection techniques. With the application of the similarity-based learning control strategy, host system can efficiently accomplish the same tasks by leveraging the successful experience of guest system, without the necessity to repeat the trial-and-error process experienced by the guest system.
Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models
Authors: Bin Fu, Qiyang Wan, Jialin Li, Ruiping Wang, Xilin Chen
Date: 2024-09-03 00:00:00
Tasks: Question Answering, Visual Question Answering

Categorization, a core cognitive ability in humans that organizes objects based on common features, is essential to cognitive science as well as computer vision. To evaluate the categorization ability of visual AI models, various proxy tasks on recognition from datasets to open world scenarios have been proposed. Recent development of Large Multimodal Models (LMMs) has demonstrated impressive results in high-level visual tasks, such as visual question answering, video temporal reasoning, etc., utilizing the advanced architectures and large-scale multimodal instruction tuning. Previous researchers have developed holistic benchmarks to measure the high-level visual capability of LMMs, but there is still a lack of pure and in-depth quantitative evaluation of the most fundamental categorization ability. According to the research on human cognitive process, categorization can be seen as including two parts: category learning and category use. Inspired by this, we propose a novel, challenging, and efficient benchmark based on composite blocks, called ComBo, which provides a disentangled evaluation framework and covers the entire categorization process from learning to use. By analyzing the results of multiple evaluation tasks, we find that although LMMs exhibit acceptable generalization ability in learning new categories, there are still gaps compared to humans in many ways, such as fine-grained perception of spatial relationship and abstract category understanding. Through the study of categorization, we can provide inspiration for the further development of LMMs in terms of interpretability and generalization.