pwc-archive/papers-with-abstracts huggingface dataset:
Distributed Differential Evolution Based on Adaptive Mergence and Split for Large-Scale Optimization
Authors: Yong-Feng Ge, Wei-Jie Yu, Ying Lin, Yue-Jiao Gong, Zhi-Hui Zhan, Wei-neng Chen, Jun Zhang et al.
Date: 2017-07-31 00:00:00
Proceeding: IEEE Transactions on Cybernetics 2017 7

Nowadays, large-scale optimization problems are ubiquitous in many research fields. To deal with such problems efficiently, this paper proposes a distributed differential evolution with adaptive mergence and split (DDE-AMS) on subpopulations. The novel mergence and split operators are designed to make full use of limited population resource, which is important for large-scale optimization. They are adaptively performed based on the performance of the subpopulations. During the evolution, once a subpopulation finds a promising region, the current worst performing subpopulation will merge into it. If the merged subpopulation could not continuously provide competitive solutions, it will be split in half. In this way, the number of subpopulations is adaptively adjusted and better performing subpopulations obtain more individuals. Thus, population resource can be adaptively arranged for subpopulations during the evolution. Moreover, the proposed algorithm is implemented with a parallel master-slave manner. Extensive experiments are conducted on 20 widely used large-scale benchmark functions. Experimental results demonstrate that the proposed DDE-AMS could achieve competitive or even better performance compared with several state-of-the-art algorithms. The effects of DDE-AMS components, adaptive behavior, scalability, and parameter sensitivity are also studied. Finally, we investigate the speedup ratios of DDE-AMS with different computation resources.
Distributed Differential Evolution Based on Adaptive Mergence and Split for Large-Scale Optimization
Authors: Yinglan Feng, Liang Feng, Senior Member, Sam Kwong, and Kay Chen Tan, Fellow, IEEE
Date: 2021-10-13 00:00:00
Proceeding: IEEE Transactions on Evolutionary Computation 2021 10

Nowadays, large-scale optimization problems are ubiquitous in many research fields. To deal with such problems efficiently, this paper proposes a distributed differential evolution with adaptive mergence and split (DDE-AMS) on subpopulations. The novel mergence and split operators are designed to make full use of limited population resource, which is important for large-scale optimization. They are adaptively performed based on the performance of the subpopulations. During the evolution, once a subpopulation finds a promising region, the current worst performing subpopulation will merge into it. If the merged subpopulation could not continuously provide competitive solutions, it will be split in half. In this way, the number of subpopulations is adaptively adjusted and better performing subpopulations obtain more individuals. Thus, population resource can be adaptively arranged for subpopulations during the evolution. Moreover, the proposed algorithm is implemented with a parallel master-slave manner. Extensive experiments are conducted on 20 widely used large-scale benchmark functions. Experimental results demonstrate that the proposed DDE-AMS could achieve competitive or even better performance compared with several state-of-the-art algorithms. The effects of DDE-AMS components, adaptive behavior, scalability, and parameter sensitivity are also studied. Finally, we investigate the speedup ratios of DDE-AMS with different computation resources.
Effects of heavy metal exposure on hypertension: A machine learning modeling approach
Authors: Wenxiang Li 1, Guangyi Huang 1, Ningning Tang 1, Peng Lu 1, Li Jiang 1, Jian Lv, Yuanjun Qin, Yunru Lin, Fan Xu **, Daizai Lei *
Date: 2023-06-06 00:00:00
Proceeding: Chemosphere 2023 6
Tasks: Feature Importance

Heavy metal exposure is a common risk factor for hypertension. To develop an interpretable predictive machine learning (ML) model for hypertension based on levels of heavy metal exposure, data from the NHANES (2003–2016) were employed. Random forest (RF), support vector machine (SVM), decision tree (DT), multilayer perceptron (MLP), ridge regression (RR), AdaBoost (AB), gradient boosting decision tree (GBDT), voting classifier (VC), and K-nearest neighbour (KNN) algorithms were utilized to generate an optimal predictive model for hypertension. Three interpretable methods, the permutation feature importance analysis, partial dependence plot (PDP), and Shapley additive explanations (SHAP) methods, were integrated into a pipeline and embedded in ML for model interpretation. A total of 9005 eligible individuals were randomly allocated into two distinct sets for predictive model training and validation. The results showed that among the predictive models, the RF model demonstrated the highest performance, achieving an accuracy rate of 77.40% in the validation set. The AUC and F1 score for the model were 0.84 and 0.76, respectively. Blood Pb, urinary Cd, urinary Tl, and urinary Co levels were identified as the main influencers of hypertension, and their contribution weights were 0.0504 ± 0.0482, 0.0389 ± 0.0256, 0.0307 ± 0.0179, and 0.0296 ± 0.0162, respectively. Blood Pb (0.55–2.93 μg/dL) and urinary Cd (0.06–0.15 μg/L) levels exhibited the most pronounced upwards trend with the risk of hypertension within a specific value range, while urinary Tl (0.06–0.26 μg/L) and urinary Co (0.02–0.32 μg/L) levels demonstrated a declining trend with hypertension. The findings on the synergistic effects indicated that Pb and Cd were the primary determinants of hypertension. Our findings underscore the predictive value of heavy metals for hy pertension. By utilizing interpretable methods, we discerned that Pb, Cd, Tl, and Co emerged as noteworthy contributors within the predictive model.
PGDQN: Preference-Guided Deep Q-Network
Authors: Wenhui Huang, Cong Zhang, Jingda Wu, Xiangkun He, Jie Zhang, Chen Lv.
Date: 2023-10-03 00:00:00
Proceeding: IEEE Transactions on Neural Networks and Learning Systems 2023 10
Tasks: Atari Games, Benchmarking, Efficient Exploration, Q-Learning, Reinforcement Learning (RL)

Stochastic exploration is the key to the success of the Deep Q-network (DQN) algorithm. However, most existing stochastic exploration approaches either explore actions heuristically regardless of their Q-values or couple the sampling with Q-values which inevitably introduce bias into the learning process. In this paper, we propose a novel preference-guided $\epsilon$-greedy exploration algorithm that can efficiently facilitate exploration for DQN without introducing additional bias. Specifically, we design a dual architecture consisting of two branches, one of which is a copy of DQN, namely the Q-branch. The other branch, which we call the preference branch, learns the action preference that the DQN implicitly follows. We theoretically prove that the policy improvement theorem holds for the preference-guided $\epsilon$-greedy policy and experimentally show that the inferred action preference distribution aligns with the landscape of corresponding Q-values. Intuitively, the preference-guided $\epsilon$-greedy exploration motivates the DQN agent to take diverse actions so that actions with larger Q-values can be sampled more frequently, and those with smaller Q-values still have a chance to be explored, thus encouraging the exploration. We comprehensively evaluate the proposed method by benchmarking it with well-known DQN variants in nine different environments. Extensive results confirm the superiority of our proposed method in terms of performance and convergence speed. The demonstrative video and source code are available at \url{https://github.com/OscarHuangWind/Preference-Guided-DQN-Atari}.
Sheet Music Reader
Authors: Sevy Harris, Prateek Verma
Date: 2015-01-01 00:00:00
Proceeding: Stanford 2015 1

The goal of this project was to design an image processing algorithm that scans in sheet music and plays the
music described on the page. Our algorithm takes a digital image of sheet music and segments it by line. For each line,
it detects the note types and locations, and computes the frequency and duration. These frequency and duration pairs
are stored in a matrix and fed into the audio synthesizer, which plays back the music. For simple and non-noisy input
images, the audio output for this algorithm was recognizable and played the pitches accurately, excluding the presence
of accidentals.
SemStamp: A Semantic Watermark with Paraphrastic Robustness for Text Generation
Authors: Abe Bohan Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, Yulia Tsvetkov
Date: 2023-10-06 00:00:00
Tasks: Sentence, Text Generation

Existing watermarking algorithms are vulnerable to paraphrase attacks because of their token-level design. To address this issue, we propose SemStamp, a robust sentence-level semantic watermarking algorithm based on locality-sensitive hashing (LSH), which partitions the semantic space of sentences. The algorithm encodes and LSH-hashes a candidate sentence generated by an LLM, and conducts sentence-level rejection sampling until the sampled sentence falls in watermarked partitions in the semantic embedding space. A margin-based constraint is used to enhance its robustness. To show the advantages of our algorithm, we propose a "bigram" paraphrase attack using the paraphrase that has the fewest bigram overlaps with the original sentence. This attack is shown to be effective against the existing token-level watermarking method. Experimental results show that our novel semantic watermark algorithm is not only more robust than the previous state-of-the-art method on both common and bigram paraphrase attacks, but also is better at preserving the quality of generation.
Investigating Alternative Feature Extraction Pipelines For Clinical Note Phenotyping
Authors: Neil Daniel
Date: 2023-10-05 00:00:00
Tasks: Clinical Note Phenotyping

A common practice in the medical industry is the use of clinical notes, which consist of detailed patient observations. However, electronic health record systems frequently do not contain these observations in a structured format, rendering patient information challenging to assess and evaluate automatically. Using computational systems for the extraction of medical attributes offers many applications, including longitudinal analysis of patients, risk assessment, and hospital evaluation. Recent work has constructed successful methods for phenotyping: extracting medical attributes from clinical notes. BERT-based models can be used to transform clinical notes into a series of representations, which are then condensed into a single document representation based on their CLS embeddings and passed into an LSTM (Mulyar et al., 2020). Though this pipeline yields a considerable performance improvement over previous results, it requires extensive convergence time. This method also does not allow for predicting attributes not yet identified in clinical notes. Considering the wide variety of medical attributes that may be present in a clinical note, we propose an alternative pipeline utilizing ScispaCy (Neumann et al., 2019) for the extraction of common diseases. We then train various supervised learning models to associate the presence of these conditions with patient attributes. Finally, we replicate a ClinicalBERT (Alsentzer et al., 2019) and LSTM-based approach for purposes of comparison. We find that alternative methods moderately underperform the replicated LSTM approach. Yet, considering a complex tradeoff between accuracy and runtime, in addition to the fact that the alternative approach also allows for the detection of medical conditions that are not already present in a clinical note, its usage may be considered as a supplement to established methods.
Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
Authors: Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, Hang Zhao
Date: 2023-10-06 00:00:00
Tasks: GPU, Image Generation, Text to Image Generation, Text-to-Image Generation

Latent Diffusion models (LDMs) have achieved remarkable results in synthesizing high-resolution images. However, the iterative sampling process is computationally intensive and leads to slow generation. Inspired by Consistency Models (song et al.), we propose Latent Consistency Models (LCMs), enabling swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion (rombach et al). Viewing the guided reverse diffusion process as solving an augmented probability flow ODE (PF-ODE), LCMs are designed to directly predict the solution of such ODE in latent space, mitigating the need for numerous iterations and allowing rapid, high-fidelity sampling. Efficiently distilled from pre-trained classifier-free guided diffusion models, a high-quality 768 x 768 2~4-step LCM takes only 32 A100 GPU hours for training. Furthermore, we introduce Latent Consistency Fine-tuning (LCF), a novel method that is tailored for fine-tuning LCMs on customized image datasets. Evaluation on the LAION-5B-Aesthetics dataset demonstrates that LCMs achieve state-of-the-art text-to-image generation performance with few-step inference. Project Page: https://latent-consistency-models.github.io/
Improved prediction of ligand-protein binding affinities by meta-modeling
Authors: Ho-Joon Lee, Prashant S. Emani, Mark B. Gerstein
Date: 2023-10-05 00:00:00

The accurate screening of candidate drug ligands against target proteins through computational approaches is of prime interest to drug development efforts. Such virtual screening depends in part on methods to predict the binding affinity between ligands and proteins. Many computational models for binding affinity prediction have been developed, but with varying results across targets. Given that ensembling or meta-modeling approaches have shown great promise in reducing model-specific biases, we develop a framework to integrate published force-field-based empirical docking and sequence-based deep learning models. In building this framework, we evaluate many combinations of individual base models, training databases, and several meta-modeling approaches. We show that many of our meta-models significantly improve affinity predictions over base models. Our best meta-models achieve comparable performance to state-of-the-art deep learning tools exclusively based on 3D structures, while allowing for improved database scalability and flexibility through the explicit inclusion of features such as physicochemical properties or molecular descriptors. We further demonstrate improved generalization capability by our models using a large-scale benchmark of affinity prediction as well as a virtual screening application benchmark. Overall, we demonstrate that diverse modeling approaches can be ensembled together to gain meaningful improvement in binding affinity prediction.
Feature Hashing for Large Scale Multitask Learning
Authors: Kilian Weinberger, Anirban Dasgupta, Josh Attenberg, John Langford, Alex Smola
Date: 2009-02-12 00:00:00
Tasks: Dimensionality Reduction

Empirical evidence suggests that hashing is an effective strategy for dimensionality reduction and practical nonparametric estimation. In this paper we provide exponential tail bounds for feature hashing and show that the interaction between random subspaces is negligible with high probability. We demonstrate the feasibility of this approach with experimental results for a new use case -- multitask learning with hundreds of thousands of tasks.
Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models
Authors: Boyu Zhang, Hongyang Yang, Tianyu Zhou, Ali Babar, Xiao-Yang Liu
Date: 2023-10-06 00:00:00
Tasks: Decision Making, Retrieval, Sentiment Analysis

Financial sentiment analysis is critical for valuation and investment decision-making. Traditional NLP models, however, are limited by their parameter size and the scope of their training datasets, which hampers their generalization capabilities and effectiveness in this field. Recently, Large Language Models (LLMs) pre-trained on extensive corpora have demonstrated superior performance across various NLP tasks due to their commendable zero-shot abilities. Yet, directly applying LLMs to financial sentiment analysis presents challenges: The discrepancy between the pre-training objective of LLMs and predicting the sentiment label can compromise their predictive performance. Furthermore, the succinct nature of financial news, often devoid of sufficient context, can significantly diminish the reliability of LLMs' sentiment analysis. To address these challenges, we introduce a retrieval-augmented LLMs framework for financial sentiment analysis. This framework includes an instruction-tuned LLMs module, which ensures LLMs behave as predictors of sentiment labels, and a retrieval-augmentation module which retrieves additional context from reliable external sources. Benchmarked against traditional models and LLMs like ChatGPT and LLaMA, our approach achieves 15\% to 48\% performance gain in accuracy and F1 score.
KoMultiText: Large-Scale Korean Text Dataset for Classifying Biased Speech in Real-World Online Services
Authors: Dasol Choi, Jooyoung Song, Eunsun Lee, JinWoo Seo, Heejune Park, Dongbin Na
Date: 2023-10-06 00:00:00
Tasks: Hate Speech Detection, Multi-Task Learning, Sentiment Analysis, text-classification, Text Classification, Text Detection

With the growth of online services, the need for advanced text classification algorithms, such as sentiment analysis and biased text detection, has become increasingly evident. The anonymous nature of online services often leads to the presence of biased and harmful language, posing challenges to maintaining the health of online communities. This phenomenon is especially relevant in South Korea, where large-scale hate speech detection algorithms have not yet been broadly explored. In this paper, we introduce "KoMultiText", a new comprehensive, large-scale dataset collected from a well-known South Korean SNS platform. Our proposed dataset provides annotations including (1) Preferences, (2) Profanities, and (3) Nine types of Bias for the text samples, enabling multi-task learning for simultaneous classification of user-generated texts. Leveraging state-of-the-art BERT-based language models, our approach surpasses human-level accuracy across diverse classification tasks, as measured by various metrics. Beyond academic contributions, our work can provide practical solutions for real-world hate speech and bias mitigation, contributing directly to the improvement of online community health. Our work provides a robust foundation for future research aiming to improve the quality of online discourse and foster societal well-being. All source codes and datasets are publicly accessible at https://github.com/Dasol-Choi/KoMultiText.
BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity
Authors: Andrew F. Luo, Margaret M. Henderson, Michael J. Tarr, Leila Wehbe
Date: 2023-10-06 00:00:00
Tasks: Image Generation, Language Modeling, Language Modelling, Large Language Model, scientific discovery

Understanding the functional organization of higher visual cortex is a central focus in neuroscience. Past studies have primarily mapped the visual and semantic selectivity of neural populations using hand-selected stimuli, which may potentially bias results towards pre-existing hypotheses of visual cortex functionality. Moving beyond conventional approaches, we introduce a data-driven method that generates natural language descriptions for images predicted to maximally activate individual voxels of interest. Our method -- Semantic Captioning Using Brain Alignments ("BrainSCUBA") -- builds upon the rich embedding space learned by a contrastive vision-language model and utilizes a pre-trained large language model to generate interpretable captions. We validate our method through fine-grained voxel-level captioning across higher-order visual regions. We further perform text-conditioned image synthesis with the captions, and show that our images are semantically coherent and yield high predicted activations. Finally, to demonstrate how our method enables scientific discovery, we perform exploratory investigations on the distribution of "person" representations in the brain, and discover fine-grained semantic selectivity in body-selective areas. Unlike earlier studies that decode text, our method derives voxel-wise captions of semantic selectivity. Our results show that BrainSCUBA is a promising means for understanding functional preferences in the brain, and provides motivation for further hypothesis-driven investigation of visual cortex.
Functional Interpolation for Relative Positions Improves Long Context Transformers
Authors: Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontanon, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli
Date: 2023-10-06 00:00:00
Tasks: Language Modeling, Language Modelling, Position

Preventing the performance decay of Transformers on inputs longer than those used for training has been an important challenge in extending the context length of these models. Though the Transformer architecture has fundamentally no limits on the input sequence lengths it can process, the choice of position encoding used during training can limit the performance of these models on longer inputs. We propose a novel functional relative position encoding with progressive interpolation, FIRE, to improve Transformer generalization to longer contexts. We theoretically prove that this can represent some of the popular relative position encodings, such as T5's RPE, Alibi, and Kerple. We next empirically show that FIRE models have better generalization to longer contexts on both zero-shot language modeling and long text benchmarks.
Diffusion Random Feature Model
Authors: Esha Saha, Giang Tran
Date: 2023-10-06 00:00:00
Tasks: Generalization Bounds, model

Diffusion probabilistic models have been successfully used to generate data from noise. However, most diffusion models are computationally expensive and difficult to interpret with a lack of theoretical justification. Random feature models on the other hand have gained popularity due to their interpretability but their application to complex machine learning tasks remains limited. In this work, we present a diffusion model-inspired deep random feature model that is interpretable and gives comparable numerical results to a fully connected neural network having the same number of trainable parameters. Specifically, we extend existing results for random features and derive generalization bounds between the distribution of sampled data and the true distribution using properties of score matching. We validate our findings by generating samples on the fashion MNIST dataset and instrumental audio data.