pwc-archive/papers-with-abstracts huggingface dataset:
Online Uniform Sampling: Randomized Learning-Augmented Approximation Algorithms with Application to Digital Health
Authors: Xueqing Liu, Kyra Gan, Esmaeil Keyvanshokooh, Susan Murphy
Date: 2024-02-03 00:00:00

Motivated by applications in digital health, this work studies the novel problem of online uniform sampling (OUS), where the goal is to distribute a sampling budget uniformly across unknown decision times. In the OUS problem, the algorithm is given a budget $b$ and a time horizon $T$, and an adversary then chooses a value $\tau^* \in [b,T]$, which is revealed to the algorithm online. At each decision time $i \in [\tau^*]$, the algorithm must determine a sampling probability that maximizes the budget spent throughout the horizon, respecting budget constraint $b$, while achieving as uniform a distribution as possible over $\tau^*$. We present the first randomized algorithm designed for this problem and subsequently extend it to incorporate learning augmentation. We provide worst-case approximation guarantees for both algorithms, and illustrate the utility of the algorithms through both synthetic experiments and a real-world case study involving the HeartSteps mobile application. Our numerical results show strong empirical average performance of our proposed randomized algorithms against previously proposed heuristic solutions.
Robust support vector machines via conic optimization
Authors: Valentina Cepeda, Andrés Gómez, Shaoning Han
Date: 2024-02-02 00:00:00

We consider the problem of learning support vector machines robust to uncertainty. It has been established in the literature that typical loss functions, including the hinge loss, are sensible to data perturbations and outliers, thus performing poorly in the setting considered. In contrast, using the 0-1 loss or a suitable non-convex approximation results in robust estimators, at the expense of large computational costs. In this paper we use mixed-integer optimization techniques to derive a new loss function that better approximates the 0-1 loss compared with existing alternatives, while preserving the convexity of the learning problem. In our computational results, we show that the proposed estimator is competitive with the standard SVMs with the hinge loss in outlier-free regimes and better in the presence of outliers.
GraphRT: A graph-based deep learning model for predicting the retention time of peptides
Authors: Mark Drvodelic, Mingming Gong, Andrew I. Webb
Date: 2024-02-05 00:00:00
Tasks: Graph Neural Network

GraphRT is a graph based deep learning model that predicts the retention time (RT) of peptides in liquid chromatography tandem mass spectrometry (LC MSMS) experiments. Each amino acid is represented as a graph, capturing its atomic and structural properties through a graph neural network. This enables the model to understand not just the chemical composition of each amino acid, but also the intricate relationships between its atoms. The sequential context of the peptide the order and interaction of amino acids in the sequence is then encoded using recurrent neural networks. This dual approach of graph based and sequential modelling allows for a comprehensive understanding of both the individual characteristics of amino acids and their collective behaviour in a peptide sequence. GraphRT outperforms all current state of the art models and can predict retention time for peptides containing unseen modifications.
Enhancing the efficiency of protein language models with minimal wet-lab data through few-shot learning
Authors: Ziyi Zhou, Liang Zhang, Yuanxi Yu, Mingchen Li, Liang Hong, Pan Tan
Date: 2024-02-03 00:00:00
Tasks: Few-Shot Learning, Learning-To-Rank, parameter-efficient fine-tuning, Protein Design, Transfer Learning

Accurately modeling the protein fitness landscapes holds great importance for protein engineering. Recently, due to their capacity and representation ability, pre-trained protein language models have achieved state-of-the-art performance in predicting protein fitness without experimental data. However, their predictions are limited in accuracy as well as interpretability. Furthermore, such deep learning models require abundant labeled training examples for performance improvements, posing a practical barrier. In this work, we introduce FSFP, a training strategy that can effectively optimize protein language models under extreme data scarcity. By combining the techniques of meta-transfer learning, learning to rank, and parameter-efficient fine-tuning, FSFP can significantly boost the performance of various protein language models using merely tens of labeled single-site mutants from the target protein. The experiments across 87 deep mutational scanning datasets underscore its superiority over both unsupervised and supervised approaches, revealing its potential in facilitating AI-guided protein design.
PhenoLinker: Phenotype-Gene Link Prediction and Explanation using Heterogeneous Graph Neural Networks
Authors: Jose L. Mellina Andreu, Luis Bernal, Antonio F. Skarmeta, Mina Ryten, Sara Álvarez, Alejandro Cisterna García, Juan A. Botía
Date: 2024-02-02 00:00:00
Tasks: Link Prediction

The association of a given human phenotype to a genetic variant remains a critical challenge for biology. We present a novel system called PhenoLinker capable of associating a score to a phenotype-gene relationship by using heterogeneous information networks and a convolutional neural network-based model for graphs, which can provide an explanation for the predictions. This system can aid in the discovery of new associations and in the understanding of the consequences of human genetic variation.
Multiple Neuronal Specializations Elicited By Socially Driven Recognition Of Food Odors
Authors: Viktor Plusnin, Nadezhda Khoroshavkina, Ali Abonakour, Arsenii Onuchin, Viktoriya Manyukhina, Nadezhda Titova, Artem Kirsanov, Vasiliy Solodovnikov, Nikita Pospelov, Vladimir Sotskov
Date: 2024-02-05 00:00:00
Tasks: Experimental Design

This study investigates the dynamics of non-spatial specializations in hippocampal place cells during exposure to novel environments. Hippocampal place cells, known for their role in spatial mapping, exhibit multi-modal responses to sensory cues. The research focuses on understanding how these cells adapt their specialization in response to novel stimuli, specifically examining non-spatial determinants such as odors and social interactions. Using a social-driven food odor recognition model in mice, the study records CA1 hippocampal neuron activity through miniscope imaging. The experimental design involves demonstrations of novel odors to mice, followed by observation sessions with food options. The analysis employs deep neural network tools for behavior tracking and the custom-developed INTENS software package for identifying neural specializations. Results indicate multiple specializations, particularly those related to odor, with differences observed between training and testing sessions. The findings suggest a temporal aspect to the formation of these specializations in novel conditions, necessitating further investigation for precise tracking.
The explanatory power of activity flow models of brain function
Authors: Michael W. Cole
Date: 2024-02-03 00:00:00

Tremendous neuroscientific progress has recently been made by mapping brain connectivity, complementing extensive knowledge of task-evoked brain activation patterns. However, despite evidence that they are related, these connectivity and activity lines of research have mostly progressed separately. Here I review the notable productivity and future promise of combining connectivity and task-evoked activity estimates into activity flow models. These data-driven computational models simulate the generation of task-evoked activations (including those linked to behavior), producing empirically-supported explanations of the origin of neurocognitive functions based on the flow of task-evoked activity over empirical brain connections. Critically, by incorporating causal principles and extensive empirical constraints from brain data, this approach can provide more mechanistic accounts of neurocognitive phenomena than purely predictive (as opposed to explanatory) models or models optimized primarily for task performance (e.g., standard artificial neural networks). The variety of activity-flow-based explanations reported so far are covered here along with important methodological and theoretical considerations when discovering new activity-flow-based explanations. Together, these considerations illustrate the promise of activity flow modeling for the future of neuroscience and ultimately for the development of novel clinical treatments (e.g., using brain stimulation) for brain disorders.
Optimize Individualized Energy Delivery for Septic Patients Using Predictive Deep Learning Models: A Real World Study
Authors: Lu Wang, Li Chang, Ruipeng Zhang, Kexun Li, Yu Wang, Wei Chen, Xuanlin Feng, Mingwei Sun, Qi Wang, Charles Damien Lu, Jun Zeng, Hua Jiang
Date: 2024-02-03 00:00:00
Tasks: Nutrition, Time Series Prediction

Background and Objectives: We aim to establish deep learning models to optimize the individualized energy delivery for septic patients. Methods and Study Design: We conducted a study of adult septic patients in Intensive Care Unit (ICU), collecting 47 indicators for 14 days. After data cleaning and preprocessing, we used stats to explore energy delivery in deceased and surviving patients. We filtered out nutrition-related features and divided the data into three metabolic phases: acute early, acute late, and rehabilitation. Models were built using data before September 2020 and validated on the rest. We then established optimal energy target models for each phase using deep learning. Results: A total of 277 patients and 3115 data were included in this study. The models indicated that the optimal energy targets in the three phases were 900kcal/d, 2300kcal/d, and 2000kcal/d, respectively. Excessive energy intake increased mortality rapidly in the early period of the acute phase. Insufficient energy in the late period of the acute phase significantly raised the mortality of septic patients. For the rehabilitation phase, too much or too little energy delivery both associated with high mortality. Conclusion: Our study established time-series prediction models for septic patients to optimize energy delivery in the ICU. This approach indicated the feasibility of developing nutritional tools for critically ill patients. We recommended permissive underfeeding only in the early acute phase. Later, increased energy intake may improve survival and settle energy debts caused by underfeeding.
Cell Painting Gallery: an open resource for image-based profiling
Authors: Erin Weisbart, Ankur Kumar, John Arevalo, Anne E. Carpenter, Beth A. Cimini, Shantanu Singh
Date: 2024-02-03 00:00:00

Image-based or morphological profiling is a rapidly expanding field wherein cells are "profiled" by extracting hundreds to thousands of unbiased, quantitative features from images of cells that have been perturbed by genetic or chemical perturbations. The Cell Painting assay is the most popular imaged-based profiling assay wherein six small-molecule dyes label eight cellular compartments and thousands of measurements are made, describing quantitative traits such as size, shape, intensity, and texture within the nucleus, cytoplasm, and whole cell (Cimini et al., 2023). We have created the Cell Painting Gallery, a publicly available collection of Cell Painting datasets, with granular dataset descriptions and access instructions. It is hosted by AWS on the Registry of Open Data (RODA). As of January 2024, the Cell Painting Gallery holds 656 terabytes (TB) of image and associated numerical data. It includes the largest publicly available Cell Painting dataset, in terms of perturbations tested (Joint Undertaking for Morphological Profiling or JUMP (Chandrasekaran et al., 2023)), along with many other canonical datasets using Cell Painting, close derivatives of Cell Painting (such as LipocyteProfiler (Laber et al., 2023) and Pooled Cell Painting (Ramezani et al., 2023)).
CFTM: Continuous time fractional topic model
Authors: Kei Nakagawa, Kohei Hayashi, Yugo Fujimoto
Date: 2024-01-29 00:00:00
Tasks: Articles, Dynamic Topic Modeling, model, parameter estimation, Topic Models

In this paper, we propose the Continuous Time Fractional Topic Model (cFTM), a new method for dynamic topic modeling. This approach incorporates fractional Brownian motion~(fBm) to effectively identify positive or negative correlations in topic and word distribution over time, revealing long-term dependency or roughness. Our theoretical analysis shows that the cFTM can capture these long-term dependency or roughness in both topic and word distributions, mirroring the main characteristics of fBm. Moreover, we prove that the parameter estimation process for the cFTM is on par with that of LDA, traditional topic models. To demonstrate the cFTM's property, we conduct empirical study using economic news articles. The results from these tests support the model's ability to identify and track long-term dependency or roughness in topics over time.
Signature volatility models: pricing and hedging with Fourier
Authors: Eduardo Abi Jaber, Louis-Amand Gérard
Date: 2024-02-02 00:00:00
Tasks: tensor algebra

We consider a stochastic volatility model where the dynamics of the volatility are given by a possibly infinite linear combination of the elements of the time extended signature of a Brownian motion. First, we show that the model is remarkably universal, as it includes, but is not limited to, the celebrated Stein-Stein, Bergomi, and Heston models, together with some path-dependent variants. Second, we derive the joint characteristic functional of the log-price and integrated variance provided that some infinite dimensional extended tensor algebra valued Riccati equation admits a solution. This allows us to price and (quadratically) hedge certain European and path-dependent options using Fourier inversion techniques. We highlight the efficiency and accuracy of these Fourier techniques in a comprehensive numerical study.
Optimal dynamic climate adaptation pathways: a case study of New York City
Authors: Chi Truong, Matteo Malavasi, Han Li, Stefan Trueck, Pavel V. Shevchenko
Date: 2024-02-05 00:00:00

Assessing climate risk and its potential impacts on our cities and economies is of fundamental importance. Extreme weather events, such as hurricanes, floods, and storm surges can lead to catastrophic damages. We propose a flexible approach based on real options analysis and extreme value theory, which enables the selection of optimal adaptation pathways for a portfolio of climate adaptation projects. We model the severity of extreme sea level events using the block maxima approach from extreme value theory, and then develop a real options framework, factoring in climate change, sea level rise uncertainty, and the growth in asset exposure. We then apply the proposed framework to a real-world problem, considering sea level data as well as different adaptation investment options for New York City. Our research can assist governments and policy makers in taking informed decisions about optimal adaptation pathways and more specifically about reducing flood and storm surge risk in a dynamic settings.
Sample-Efficient "Clustering and Conquer" Procedures for Parallel Large-Scale Ranking and Selection
Authors: Zishi Zhang, Yijie Peng
Date: 2024-02-03 00:00:00
Tasks: Clustering, Neural Architecture Search

This work seeks to break the sample efficiency bottleneck in parallel large-scale ranking and selection (R&S) problems by leveraging correlation information. We modify the commonly used "divide and conquer" framework in parallel computing by adding a correlation-based clustering step, transforming it into "clustering and conquer". This seemingly simple modification achieves the optimal sample complexity reduction for a widely used class of efficient large-scale R&S procedures. Our approach enjoys two key advantages: 1) it does not require highly accurate correlation estimation or precise clustering, and 2) it allows for seamless integration with various existing R&S procedures, while achieving optimal sample complexity. Theoretically, we develop a novel gradient analysis framework to analyze sample efficiency and guide the design of large-scale R&S procedures. We also introduce a new parallel clustering algorithm tailored for large-scale scenarios. Finally, in large-scale AI applications such as neural architecture search, our methods demonstrate superior performance.
L-TUNING: Synchronized Label Tuning for Prompt and Prefix in LLMs
Authors: Md. Kowsher, Md. Shohanur Islam Sobuj, Asif Mahmud, Nusrat Jahan Prottasha, Prakash Bhat
Date: 2023-12-21 00:00:00
Tasks: Natural Language Inference

Efficiently fine-tuning Large Language Models (LLMs) for specific tasks presents a considerable challenge in natural language processing. Traditional methods, like prompt or prefix tuning, typically rely on arbitrary tokens for training, leading to prolonged training times and generalized token use across various class labels. To address these issues, this paper introduces L-Tuning, an efficient fine-tuning approach designed for classification tasks within the Natural Language Inference (NLI) framework. Diverging from conventional methods, L-Tuning focuses on the fine-tuning of label tokens processed through a pre-trained LLM, thereby harnessing its pre-existing semantic knowledge. This technique not only improves the fine-tuning accuracy and efficiency but also facilitates the generation of distinct label embeddings for each class, enhancing the model's training nuance. Our experimental results indicate a significant improvement in training efficiency and classification accuracy with L-Tuning compared to traditional approaches, marking a promising advancement in fine-tuning LLMs for complex language tasks.
Language models align with human judgments on key grammatical constructions
Authors: Jennifer Hu, Kyle Mahowald, Gary Lupyan, Anna Ivanova, Roger Levy
Date: 2024-01-19 00:00:00
Tasks: Sentence

Do large language models (LLMs) make human-like linguistic generalizations? Dentella et al. (2023) ("DGL") prompt several LLMs ("Is the following sentence grammatically correct in English?") to elicit grammaticality judgments of 80 English sentences, concluding that LLMs demonstrate a "yes-response bias" and a "failure to distinguish grammatical from ungrammatical sentences". We re-evaluate LLM performance using well-established practices and find that DGL's data in fact provide evidence for just how well LLMs capture human behaviors. Models not only achieve high accuracy overall, but also capture fine-grained variation in human linguistic judgments.
Emojis Decoded: Leveraging ChatGPT for Enhanced Understanding in Social Media Communications
Authors: YuHang Zhou, Paiheng Xu, Xiyao Wang, Xuan Lu, Ge Gao, Wei Ai
Date: 2024-01-22 00:00:00

Emojis, which encapsulate semantics beyond mere words or phrases, have become prevalent in social network communications. This has spurred increasing scholarly interest in exploring their attributes and functionalities. However, emoji-related research and application face two primary challenges. First, researchers typically rely on crowd-sourcing to annotate emojis in order to understand their sentiments, usage intentions, and semantic meanings. Second, subjective interpretations by users can often lead to misunderstandings of emojis and cause the communication barrier. Large Language Models (LLMs) have achieved significant success in various annotation tasks, with ChatGPT demonstrating expertise across multiple domains. In our study, we assess ChatGPT's effectiveness in handling previously annotated and downstream tasks. Our objective is to validate the hypothesis that ChatGPT can serve as a viable alternative to human annotators in emoji research and that its ability to explain emoji meanings can enhance clarity and transparency in online communications. Our findings indicate that ChatGPT has extensive knowledge of emojis. It is adept at elucidating the meaning of emojis across various application scenarios and demonstrates the potential to replace human annotators in a range of tasks.
A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm
Authors: Chao Song, Zhihao Ye, Qiqiang Lin, Qiuying Peng, Jun Wang
Date: 2024-01-22 00:00:00

With the productive evolution of large language models (LLMs) in the field of natural language processing (NLP), tons of effort has been made to effectively fine-tune common pre-trained LLMs to fulfill a variety of tasks in one or multiple specific domain. In practice, there are two prevailing ways, in which the adaptation can be achieved: (i) Multiple Independent Models: Pre-trained LLMs are fine-tuned a few times independently using the corresponding training samples from each task. (ii) An Integrated Model: Samples from all tasks are employed to fine-tune a pre-trianed LLM unitedly. To address the high computing cost and seesawing issue simultaneously, we propose a unified framework that implements a 1 + N mutli-task fine-tuning pattern in LLMs using a novel Customized Gate Control (CGC) Low-rank Adaptation (LoRA) algorithm. Our work aims to take an advantage of both MTL (i.e., CGC) and PEFT (i.e., LoRA) scheme. For a given cluster of tasks, we design an innovative layer that contains two types of experts as additional trainable parameters to make LoRA be compatible with MTL. To comprehensively evaluate the proposed framework, we conduct well-designed experiments on two public datasets. The experimental results demonstrate that the unified framework with CGC-LoRA modules achieves higher evaluation scores than all benchmarks on both two datasets.
Quality of Answers of Generative Large Language Models vs Peer Patients for Interpreting Lab Test Results for Lay Patients: Evaluation Study
Authors: Zhe He, Balu Bhasuran, Qiao Jin, Shubo Tian, Karim Hanna, Cindy Shavor, Lisbeth Garcia Arguello, Patrick Murray, Zhiyong Lu
Date: 2024-01-23 00:00:00

Lab results are often confusing and hard to understand. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to get their questions answered. We aim to assess the feasibility of using LLMs to generate relevant, accurate, helpful, and unharmful responses to lab test-related questions asked by patients and to identify potential issues that can be mitigated with augmentation approaches. We first collected lab test results related question and answer data from Yahoo! Answers and selected 53 QA pairs for this study. Using the LangChain framework and ChatGPT web portal, we generated responses to the 53 questions from four LLMs including GPT-4, Meta LLaMA 2, MedAlpaca, and ORCA_mini. We first assessed the similarity of their answers using standard QA similarity-based evaluation metrics including ROUGE, BLEU, METEOR, BERTScore. We also utilized an LLM-based evaluator to judge whether a target model has higher quality in terms of relevance, correctness, helpfulness, and safety than the baseline model. Finally, we performed a manual evaluation with medical experts for all the responses to seven selected questions on the same four aspects. The results of Win Rate and medical expert evaluation both showed that GPT-4's responses achieved better scores than all the other LLM responses and human responses on all four aspects (relevance, correctness, helpfulness, and safety). However, LLM responses occasionally also suffer from a lack of interpretation in one's medical context, incorrect statements, and lack of references. We find that compared to other three LLMs and human answer from the Q&A website, GPT-4's responses are more accurate, helpful, relevant, and safer. However, there are cases which GPT-4 responses are inaccurate and not individualized. We identified a number of ways to improve the quality of LLM responses.