pwc-archive/papers-with-abstracts huggingface dataset:
Developing ChatGPT for Biology and Medicine: A Complete Review of Biomedical Question Answering
Authors: Qing Li, Lei LI, Yu Li
Date: 2024-01-15 00:00:00
Tasks: Cross-Modal Retrieval, Medical Diagnosis, Medical Question Answering, Question Answering, Reading Comprehension, Relation Extraction

ChatGPT explores a strategic blueprint of question answering (QA) in delivering medical diagnosis, treatment recommendations, and other healthcare support. This is achieved through the increasing incorporation of medical domain data via natural language processing (NLP) and multimodal paradigms. By transitioning the distribution of text, images, videos, and other modalities from the general domain to the medical domain, these techniques have expedited the progress of medical domain question answering (MDQA). They bridge the gap between human natural language and sophisticated medical domain knowledge or expert manual annotations, handling large-scale, diverse, unbalanced, or even unlabeled data analysis scenarios in medical contexts. Central to our focus is the utilizing of language models and multimodal paradigms for medical question answering, aiming to guide the research community in selecting appropriate mechanisms for their specific medical research requirements. Specialized tasks such as unimodal-related question answering, reading comprehension, reasoning, diagnosis, relation extraction, probability modeling, and others, as well as multimodal-related tasks like vision question answering, image caption, cross-modal retrieval, report summarization, and generation, are discussed in detail. Each section delves into the intricate specifics of the respective method under consideration. This paper highlights the structures and advancements of medical domain explorations against general domain methods, emphasizing their applications across different tasks and datasets. It also outlines current challenges and opportunities for future medical domain research, paving the way for continued innovation and application in this rapidly evolving field.
The Principle of Minimum Pressure Gradient: An Alternative Basis for Physics-Informed Learning of Incompressible Fluid Mechanics
Authors: Hussam Alhussein, Mohammed Daqaq
Date: 2024-01-15 00:00:00

Recent advances in the application of physics-informed learning into the field of fluid mechanics have been predominantly grounded in the Newtonian framework, primarly leveraging Navier-Stokes Equation or one of its various derivative to train a neural network. Here, we propose an alternative approach based on variational methods. The proposed approach uses the principle of minimum pressure gradient combined with the continuity constraint to train a neural network and predict the flow field in incompressible fluids. We describe the underlying principles of the proposed approach, then use a demonstrative example to illustrate its implementation and show that it reduces the computational time per training epoch when compared to the conventional approach.
Utilizing deep learning models for the identification of enhancers and super-enhancers based on genomic and epigenomic features
Authors: Zahra Ahani, Moein Shahiki Tash, Yoel Ledo Mezquita, Jason Angel
Date: 2024-01-15 00:00:00
Tasks: Emotion Recognition

This paper provides an extensive examination of a sizable dataset of English tweets focusing on nine widely recognized cryptocurrencies, specifically Cardano, Binance, Bitcoin, Dogecoin, Ethereum, Fantom, Matic, Shiba, and Ripple. Our primary objective was to conduct a psycholinguistic and emotion analysis of social media content associated with these cryptocurrencies. To enable investigators to make more informed decisions. The study involved comparing linguistic characteristics across the diverse digital coins, shedding light on the distinctive linguistic patterns that emerge within each coin's community. To achieve this, we utilized advanced text analysis techniques. Additionally, our work unveiled an intriguing Understanding of the interplay between these digital assets within the cryptocurrency community. By examining which coin pairs are mentioned together most frequently in the dataset, we established correlations between different cryptocurrencies. To ensure the reliability of our findings, we initially gathered a total of 832,559 tweets from Twitter. These tweets underwent a rigorous preprocessing stage, resulting in a refined dataset of 115,899 tweets that were used for our analysis. Overall, our research offers valuable Perception into the linguistic nuances of various digital coins' online communities and provides a deeper understanding of their interactions in the cryptocurrency space.
CarSpeedNet: A Deep Neural Network-based Car Speed Estimation from Smartphone Accelerometer
Authors: Barak Or
Date: 2024-01-15 00:00:00

We introduce the CarSpeedNet, a deep learning model designed to estimate car speed using three-axis accelerometer data from smartphones. Using 13 hours of data collected from a smartphone in cars across various roads, CarSpeedNet accurately models the relationship between smartphone acceleration and car speed. Ground truth speed data was collected at 1 [Hz] from GPS receivers. The model provides high-frequency speed estimation by incorporating historical data and achieves a precision of less than 0.72 [m/s] during extended driving tests, relying solely on smartphone accelerometer data without any connection to the car.
Your Instructions Are Not Always Helpful: Assessing the Efficacy of Instruction Fine-tuning for Software Vulnerability Detection
Authors: Imam Nur Bani Yusuf, Lingxiao Jiang
Date: 2024-01-15 00:00:00
Tasks: Deep Learning, Feature Engineering, Language Modeling, Language Modelling, Vulnerability Detection

Software, while beneficial, poses potential cybersecurity risks due to inherent vulnerabilities. Detecting these vulnerabilities is crucial, and deep learning has shown promise as an effective tool for this task due to its ability to perform well without extensive feature engineering. However, a challenge in deploying deep learning for vulnerability detection is the limited availability of training data. Recent research highlights the deep learning efficacy in diverse tasks. This success is attributed to instruction fine-tuning, a technique that remains under-explored in the context of vulnerability detection. This paper investigates the capability of models, specifically a recent language model, to generalize beyond the programming languages used in their training data. It also examines the role of natural language instructions in enhancing this generalization. Our study evaluates the model performance on a real-world dataset to predict vulnerable code. We present key insights and lessons learned, contributing to understanding the deep learning application in software vulnerability detection.
Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation
Authors: Yun-Wei Chu, Dong-Jun Han, Christopher G. Brinton
Date: 2024-01-15 00:00:00
Tasks: Federated Learning, Machine Translation, Meta-Learning, NMT, Translation

Federated learning (FL) is a promising distributed machine learning paradigm that enables multiple clients to collaboratively train a global model. In this paper, we focus on a practical federated multilingual learning setup where clients with their own language-specific data aim to collaboratively construct a high-quality neural machine translation (NMT) model. However, communication constraints in practical network systems present challenges for exchanging large-scale NMT engines between FL parties. We propose a meta-learning-based adaptive parameter selection methodology, MetaSend, that improves the communication efficiency of model transmissions from clients during FL-based multilingual NMT training. Our approach learns a dynamic threshold for filtering parameters prior to transmission without compromising the NMT model quality, based on the tensor deviations of clients between different FL rounds. Through experiments on two NMT datasets with different language distributions, we demonstrate that MetaSend obtains substantial improvements over baselines in translation quality in the presence of a limited communication budget.
Model Editing at Scale leads to Gradual and Catastrophic Forgetting
Authors: Akshat Gupta, Anurag Rao, Gopala Anumanchipalli
Date: 2024-01-15 00:00:00
Tasks: Model Editing, Specificity

Editing knowledge in large language models is an attractive capability to have which allows us to correct incorrectly learnt facts during pre-training, as well as update the model with an ever-growing list of new facts. While existing model editing techniques have shown promise, they are usually evaluated using metrics for reliability, specificity and generalization over one or few edits. We argue that for model editing to have practical utility, we must be able to make multiple edits to the same model. With this in mind, we evaluate the current model editing methods at scale, focusing on two state of the art methods: ROME and MEMIT. We find that as the model is edited sequentially with multiple facts, it continually forgets previously edited facts and the ability to perform downstream tasks. This forgetting happens in two phases -- an initial gradual but progressive forgetting phase followed by abrupt or catastrophic forgetting phase. Both gradual and catastrophic forgetting limit the usefulness of model editing methods at scale -- the former making model editing less effective as multiple edits are made to the model while the latter caps the scalability of such model editing methods. Our analysis also highlights other key limitations of ROME and MEMIT at scale. With our work, we push for the development and evaluation of model editing methods keeping scalability in mind.
HieraFashDiff: Hierarchical Fashion Design with Multi-stage Diffusion Models
Authors: Zhifeng Xie, Hao Li, Huiming Ding, Mengtian Li, Xinhan Di, Ying Cao
Date: 2024-01-15 00:00:00
Tasks: Fashion Synthesis, Image Generation

Fashion design is a challenging and complex process.Recent works on fashion generation and editing are all agnostic of the actual fashion design process, which limits their usage in practice.In this paper, we propose a novel hierarchical diffusion-based framework tailored for fashion design, coined as HieraFashDiff. Our model is designed to mimic the practical fashion design workflow, by unraveling the denosing process into two successive stages: 1) an ideation stage that generates design proposals given high-level concepts and 2) an iteration stage that continuously refines the proposals using low-level attributes. Our model supports fashion design generation and fine-grained local editing in a single framework. To train our model, we contribute a new dataset of full-body fashion images annotated with hierarchical text descriptions. Extensive evaluations show that, as compared to prior approaches, our method can generate fashion designs and edited results with higher fidelity and better prompt adherence, showing its promising potential to augment the practical fashion design workflow. Code and Dataset are available at https://github.com/haoli-zbdbc/hierafashdiff.
Formal Logic Enabled Personalized Federated Learning Through Property Inference
Authors: Ziyan An, Taylor T. Johnson, Meiyi Ma
Date: 2024-01-15 00:00:00
Tasks: Federated Learning, Formal Logic, Personalized Federated Learning

Recent advancements in federated learning (FL) have greatly facilitated the development of decentralized collaborative applications, particularly in the domain of Artificial Intelligence of Things (AIoT). However, a critical aspect missing from the current research landscape is the ability to enable data-driven client models with symbolic reasoning capabilities. Specifically, the inherent heterogeneity of participating client devices poses a significant challenge, as each client exhibits unique logic reasoning properties. Failing to consider these device-specific specifications can result in critical properties being missed in the client predictions, leading to suboptimal performance. In this work, we propose a new training paradigm that leverages temporal logic reasoning to address this issue. Our approach involves enhancing the training process by incorporating mechanically generated logic expressions for each FL client. Additionally, we introduce the concept of aggregation clusters and develop a partitioning algorithm to effectively group clients based on the alignment of their temporal reasoning properties. We evaluate the proposed method on two tasks: a real-world traffic volume prediction task consisting of sensory data from fifteen states and a smart city multi-task prediction utilizing synthetic data. The evaluation results exhibit clear improvements, with performance accuracy improved by up to 54% across all sequential prediction models.
Taec: a Manually annotated text dataset for trait and phenotype extraction and entity linking in wheat breeding literature
Authors: Claire Nédellec, Clara Sauvion, Robert Bossy, Mariya Borovikova, Louise Deléger
Date: 2024-01-15 00:00:00
Tasks: Articles, Entity Linking, named-entity-recognition, Named Entity Recognition

Wheat varieties show a large diversity of traits and phenotypes. Linking them to genetic variability is essential for shorter and more efficient wheat breeding programs. Newly desirable wheat variety traits include disease resistance to reduce pesticide use, adaptation to climate change, resistance to heat and drought stresses, or low gluten content of grains. Wheat breeding experiments are documented by a large body of scientific literature and observational data obtained in-field and under controlled conditions. The cross-referencing of complementary information from the literature and observational data is essential to the study of the genotype-phenotype relationship and to the improvement of wheat selection. The scientific literature on genetic marker-assisted selection describes much information about the genotype-phenotype relationship. However, the variety of expressions used to refer to traits and phenotype values in scientific articles is a hinder to finding information and cross-referencing it. When trained adequately by annotated examples, recent text mining methods perform highly in named entity recognition and linking in the scientific domain. While several corpora contain annotations of human and animal phenotypes, currently, no corpus is available for training and evaluating named entity recognition and entity-linking methods in plant phenotype literature. The Triticum aestivum trait Corpus is a new gold standard for traits and phenotypes of wheat. It consists of 540 PubMed references fully annotated for trait, phenotype, and species named entities using the Wheat Trait and Phenotype Ontology and the species taxonomy of the National Center for Biotechnology Information. A study of the performance of tools trained on the Triticum aestivum trait Corpus shows that the corpus is suitable for the training and evaluation of named entity recognition and linking.
GACE: Learning Graph-Based Cross-Page Ads Embedding For Click-Through Rate Prediction
Authors: Haowen Wang, Yuliang Du, Congyun Jin, Yujiao Li, Yingbo Wang, Tao Sun, Piqi Qin, Cong Fan
Date: 2024-01-15 00:00:00
Tasks: Click-Through Rate Prediction, Recommendation Systems

Predicting click-through rate (CTR) is the core task of many ads online recommendation systems, which helps improve user experience and increase platform revenue. In this type of recommendation system, we often encounter two main problems: the joint usage of multi-page historical advertising data and the cold start of new ads. In this paper, we proposed GACE, a graph-based cross-page ads embedding generation method. It can warm up and generate the representation embedding of cold-start and existing ads across various pages. Specifically, we carefully build linkages and a weighted undirected graph model considering semantic and page-type attributes to guide the direction of feature fusion and generation. We designed a variational auto-encoding task as pre-training module and generated embedding representations for new and old ads based on this task. The results evaluated in the public dataset AliEC from RecBole and the real-world industry dataset from Alipay show that our GACE method is significantly superior to the SOTA method. In the online A/B test, the click-through rate on three real-world pages from Alipay has increased by 3.6%, 2.13%, and 3.02%, respectively. Especially in the cold-start task, the CTR increased by 9.96%, 7.51%, and 8.97%, respectively.
Generalized Planning for the Abstraction and Reasoning Corpus
Authors: Chao Lei, Nir Lipovetzky, Krista A. Ehinger
Date: 2024-01-15 00:00:00
Tasks: ARC, valid

The Abstraction and Reasoning Corpus (ARC) is a general artificial intelligence benchmark that poses difficulties for pure machine learning methods due to its requirement for fluid intelligence with a focus on reasoning and abstraction. In this work, we introduce an ARC solver, Generalized Planning for Abstract Reasoning (GPAR). It casts an ARC problem as a generalized planning (GP) problem, where a solution is formalized as a planning program with pointers. We express each ARC problem using the standard Planning Domain Definition Language (PDDL) coupled with external functions representing object-centric abstractions. We show how to scale up GP solvers via domain knowledge specific to ARC in the form of restrictions over the actions model, predicates, arguments and valid structure of planning programs. Our experiments demonstrate that GPAR outperforms the state-of-the-art solvers on the object-centric tasks of the ARC, showing the effectiveness of GP and the expressiveness of PDDL to model ARC problems. The challenges provided by the ARC benchmark motivate research to advance existing GP solvers and understand new relations with other planning computational models. Code is available at github.com/you68681/GPAR.
Harnessing the Power of Beta Scoring in Deep Active Learning for Multi-Label Text Classification
Authors: Wei Tan, Ngoc Dang Nguyen, Lan Du, Wray Buntine
Date: 2024-01-15 00:00:00
Tasks: Active Learning, Multi Label Text Classification, Multi-Label Text Classification, text-classification, Text Classification

Within the scope of natural language processing, the domain of multi-label text classification is uniquely challenging due to its expansive and uneven label distribution. The complexity deepens due to the demand for an extensive set of annotated data for training an advanced deep learning model, especially in specialized fields where the labeling task can be labor-intensive and often requires domain-specific knowledge. Addressing these challenges, our study introduces a novel deep active learning strategy, capitalizing on the Beta family of proper scoring rules within the Expected Loss Reduction framework. It computes the expected increase in scores using the Beta Scoring Rules, which are then transformed into sample vector representations. These vector representations guide the diverse selection of informative samples, directly linking this process to the model's expected proper score. Comprehensive evaluations across both synthetic and real datasets reveal our method's capability to often outperform established acquisition techniques in multi-label text classification, presenting encouraging outcomes across various architectural and dataset scenarios.
A Rapid Review of Clustering Algorithms
Authors: Hui Yin, Amir Aryani, Stephen Petrie, Aishwarya Nambissan, Aland Astudillo, Shengyuan Cao
Date: 2024-01-14 00:00:00
Tasks: Clustering, Marketing

Clustering algorithms aim to organize data into groups or clusters based on the inherent patterns and similarities within the data. They play an important role in today's life, such as in marketing and e-commerce, healthcare, data organization and analysis, and social media. Numerous clustering algorithms exist, with ongoing developments introducing new ones. Each algorithm possesses its own set of strengths and weaknesses, and as of now, there is no universally applicable algorithm for all tasks. In this work, we analyzed existing clustering algorithms and classify mainstream algorithms across five different dimensions: underlying principles and characteristics, data point assignment to clusters, dataset capacity, predefined cluster numbers and application area. This classification facilitates researchers in understanding clustering algorithms from various perspectives and helps them identify algorithms suitable for solving specific tasks. Finally, we discussed the current trends and potential future directions in clustering algorithms. We also identified and discussed open challenges and unresolved issues in the field.
Noise-Aware Training of Neuromorphic Dynamic Device Networks
Authors: Luca Manneschi, Ian T. Vidamour, Kilian D. Stenning, Charles Swindells, Guru Venkat, David Griffin, Lai Gui, Daanish Sonawala, Denis Donskikh, Dana Hariga, Susan Stepney, Will R. Branford, Jack C. Gartside, Thomas Hayward, Matthew O. A. Ellis, Eleni Vasilaki
Date: 2024-01-14 00:00:00

Physical computing has the potential to enable widespread embodied intelligence by leveraging the intrinsic dynamics of complex systems for efficient sensing, processing, and interaction. While individual devices provide basic data processing capabilities, networks of interconnected devices can perform more complex and varied tasks. However, designing networks to perform dynamic tasks is challenging without physical models and accurate quantification of device noise. We propose a novel, noise-aware methodology for training device networks using Neural Stochastic Differential Equations (Neural-SDEs) as differentiable digital twins, accurately capturing the dynamics and associated stochasticity of devices with intrinsic memory. Our approach employs backpropagation through time and cascade learning, allowing networks to effectively exploit the temporal properties of physical devices. We validate our method on diverse networks of spintronic devices across temporal classification and regression benchmarks. By decoupling the training of individual device models from network training, our method reduces the required training data and provides a robust framework for programming dynamical devices without relying on analytical descriptions of their dynamics.
Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generation
Authors: Meng Cao, Lei Shu, Lei Yu, Yun Zhu, Nevan Wichers, Yinxiao Liu, Lei Meng
Date: 2024-01-14 00:00:00
Tasks: Language Modeling, Language Modelling, reinforcement-learning, Reinforcement Learning (RL), Text Generation

Reinforcement learning (RL) can align language models with non-differentiable reward signals, such as human preferences. However, a major challenge arises from the sparsity of these reward signals - typically, there is only a single reward for an entire output. This sparsity of rewards can lead to inefficient and unstable learning. To address this challenge, our paper introduces an novel framework that utilizes the critique capability of Large Language Models (LLMs) to produce intermediate-step rewards during RL training. Our method involves coupling a policy model with a critic language model, which is responsible for providing comprehensive feedback of each part of the output. This feedback is then translated into token or span-level rewards that can be used to guide the RL training process. We investigate this approach under two different settings: one where the policy model is smaller and is paired with a more powerful critic model, and another where a single language model fulfills both roles. We assess our approach on three text generation tasks: sentiment control, language model detoxification, and summarization. Experimental results show that incorporating artificial intrinsic rewards significantly improve both sample efficiency and the overall performance of the policy model, supported by both automatic and human evaluation.