pwc-archive/papers-with-abstracts huggingface dataset:
Variational EP with Probabilistic Backpropagation for Bayesian Neural Networks
Authors: Kehinde Olobatuyi
Date: 2023-03-02 00:00:00
Tasks: regression

I propose a novel approach for nonlinear Logistic regression using a two-layer neural network (NN) model structure with hierarchical priors on the network weights. I present a hybrid of expectation propagation called Variational Expectation Propagation approach (VEP) for approximate integration over the posterior distribution of the weights, the hierarchical scale parameters of the priors and zeta. Using a factorized posterior approximation I derive a computationally efficient algorithm, whose complexity scales similarly to an ensemble of independent sparse logistic models. The approach can be extended beyond standard activation functions and NN model structures to form flexible nonlinear binary predictors from multiple sparse linear models. I consider a hierarchical Bayesian model with logistic regression likelihood and a Gaussian prior distribution over the parameters called weights and hyperparameters. I work in the perspective of E step and M step for computing the approximating posterior and updating the parameters using the computed posterior respectively.
Multi-Start Team Orienteering Problem for UAS Mission Re-Planning with Data-Efficient Deep Reinforcement Learning
Authors: Dong Ho Lee, Jaemyung Ahn
Date: 2023-03-02 00:00:00
Tasks: Decoder, Deep Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we study the Multi-Start Team Orienteering Problem (MSTOP), a mission re-planning problem where vehicles are initially located away from the depot and have different amounts of fuel. We consider/assume the goal of multiple vehicles is to travel to maximize the sum of collected profits under resource (e.g., time, fuel) consumption constraints. Such re-planning problems occur in a wide range of intelligent UAS applications where changes in the mission environment force the operation of multiple vehicles to change from the original plan. To solve this problem with deep reinforcement learning (RL), we develop a policy network with self-attention on each partial tour and encoder-decoder attention between the partial tour and the remaining nodes. We propose a modified REINFORCE algorithm where the greedy rollout baseline is replaced by a local mini-batch baseline based on multiple, possibly non-duplicate sample rollouts. By drawing multiple samples per training instance, we can learn faster and obtain a stable policy gradient estimator with significantly fewer instances. The proposed training algorithm outperforms the conventional greedy rollout baseline, even when combined with the maximum entropy objective.
Artificial Intelligence for Dementia Research Methods Optimization
Authors: Magda Bucholc, Charlotte James, Ahmad Al Khleifat, AmanPreet Badhwar, Natasha Clarke, Amir Dehsarvi, Christopher R. Madan, Sarah J. Marzi, Cameron Shand, Brian M. Schilder, Stefano Tamburin, Hanz M. Tantiangco, Ilianna Lourida, David J. Llewellyn, Janice M. Ranson
Date: 2023-03-02 00:00:00
Tasks: Multi-Task Learning, Transfer Learning

Introduction: Machine learning (ML) has been extremely successful in identifying key features from high-dimensional datasets and executing complicated tasks with human expert levels of accuracy or greater. Methods: We summarize and critically evaluate current applications of ML in dementia research and highlight directions for future research. Results: We present an overview of ML algorithms most frequently used in dementia research and highlight future opportunities for the use of ML in clinical practice, experimental medicine, and clinical trials. We discuss issues of reproducibility, replicability and interpretability and how these impact the clinical applicability of dementia research. Finally, we give examples of how state-of-the-art methods, such as transfer learning, multi-task learning, and reinforcement learning, may be applied to overcome these issues and aid the translation of research to clinical practice in the future. Discussion: ML-based models hold great promise to advance our understanding of the underlying causes and pathological mechanisms of dementia.
Defending against Adversarial Audio via Diffusion Model
Authors: Shutong Wu, Jiongxiao Wang, Wei Ping, Weili Nie, Chaowei Xiao
Date: 2023-03-02 00:00:00
Tasks: Adversarial Purification, model

Deep learning models have been widely used in commercial acoustic systems in recent years. However, adversarial audio examples can cause abnormal behaviors for those acoustic systems, while being hard for humans to perceive. Various methods, such as transformation-based defenses and adversarial training, have been proposed to protect acoustic systems from adversarial attacks, but they are less effective against adaptive attacks. Furthermore, directly applying the methods from the image domain can lead to suboptimal results because of the unique properties of audio data. In this paper, we propose an adversarial purification-based defense pipeline, AudioPure, for acoustic systems via off-the-shelf diffusion models. Taking advantage of the strong generation ability of diffusion models, AudioPure first adds a small amount of noise to the adversarial audio and then runs the reverse sampling step to purify the noisy audio and recover clean audio. AudioPure is a plug-and-play method that can be directly applied to any pretrained classifier without any fine-tuning or re-training. We conduct extensive experiments on speech command recognition task to evaluate the robustness of AudioPure. Our method is effective against diverse adversarial attacks (e.g. $\mathcal{L}_2$ or $\mathcal{L}_\infty$-norm). It outperforms the existing methods under both strong adaptive white-box and black-box attacks bounded by $\mathcal{L}_2$ or $\mathcal{L}_\infty$-norm (up to +20\% in robust accuracy). Besides, we also evaluate the certified robustness for perturbations bounded by $\mathcal{L}_2$-norm via randomized smoothing. Our pipeline achieves a higher certified accuracy than baselines.
A Lifted Bregman Formulation for the Inversion of Deep Neural Networks
Authors: Xiaoyu Wang, Martin Benning
Date: 2023-03-01 00:00:00

We propose a novel framework for the regularised inversion of deep neural networks. The framework is based on the authors' recent work on training feed-forward neural networks without the differentiation of activation functions. The framework lifts the parameter space into a higher dimensional space by introducing auxiliary variables, and penalises these variables with tailored Bregman distances. We propose a family of variational regularisations based on these Bregman distances, present theoretical results and support their practical application with numerical examples. In particular, we present the first convergence result (to the best of our knowledge) for the regularised inversion of a single-layer perceptron that only assumes that the solution of the inverse problem is in the range of the regularisation operator, and that shows that the regularised inverse provably converges to the true inverse if measurement errors converge to zero.
Who could be behind QAnon? Authorship attribution with supervised machine-learning
Authors: Florian Cafiero, Jean-Baptiste Camps
Date: 2023-03-03 00:00:00
Tasks: Authorship Attribution

A series of social media posts signed under the pseudonym "Q", started a movement known as QAnon, which led some of its most radical supporters to violent and illegal actions. To identify the person(s) behind Q, we evaluate the coincidence between the linguistic properties of the texts written by Q and to those written by a list of suspects provided by journalistic investigation. To identify the authors of these posts, serious challenges have to be addressed. The "Q drops" are very short texts, written in a way that constitute a sort of literary genre in itself, with very peculiar features of style. These texts might have been written by different authors, whose other writings are often hard to find. After an online ethnology of the movement, necessary to collect enough material written by these thirteen potential authors, we use supervised machine learning to build stylistic profiles for each of them. We then performed a rolling analysis on Q's writings, to see if any of those linguistic profiles match the so-called 'QDrops' in part or entirety. We conclude that two different individuals, Paul F. and Ron W., are the closest match to Q's linguistic signature, and they could have successively written Q's texts. These potential authors are not high-ranked personality from the U.S. administration, but rather social media activists.
Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM
Authors: Rachel Bawden, François Yvon
Date: 2023-03-03 00:00:00
Tasks: Cross-Lingual Transfer, Language Modeling, Language Modelling, Machine Translation, Translation

The NLP community recently saw the release of a new large open-access multilingual language model, BLOOM (BigScience et al., 2022) covering 46 languages. We focus on BLOOM's multilingual ability by evaluating its machine translation performance across several datasets (WMT, Flores-101 and DiaBLa) and language pairs (high- and low-resourced). Our results show that 0-shot performance suffers from overgeneration and generating in the wrong language, but this is greatly improved in the few-shot setting, with very good results for a number of language pairs. We study several aspects including prompt design, model sizes, cross-lingual transfer and the use of discursive context.
Mapping Wordnets on the Fly with Permanent Sense Keys
Authors: Eric Kafe
Date: 2023-03-03 00:00:00

Most of the major databases on the semantic web have links to Princeton WordNet (PWN) synonym set (synset) identifiers, which differ for each PWN release, and are thus incompatible between versions. On the other hand, both PWN and the more recent Open English Wordnet (OEWN) provide permanent word sense identifiers (the sense keys), which can solve this interoperability problem. We present an algorithm that runs in linear time, to automatically derive a synset mapping between any pair of Wordnet versions that use PWN sense keys. This allows to update old WordNet links, and seamlessly interoperate with newer English Wordnet versions for which no prior mapping exists. By applying the proposed algorithm on the fly, at load time, we combine the Open Multilingual Wordnet (OMW 1.4, which uses old PWN 3.0 identifiers) with OEWN Edition 2021, and obtain almost perfect precision and recall. We compare the results of our approach using respectively synset offsets, versus the Collaborative InterLingual Index (CILI version 1.0) as synset identifiers, and find that the synset offsets perform better than CILI 1.0 in all cases, except a few ties.
PAGE: A Position-Aware Graph-Based Model for Emotion Cause Entailment in Conversation
Authors: Xiaojie Gu, Renze Lou, Lin Sun, Shangxin Li
Date: 2023-03-03 00:00:00
Tasks: Causal Emotion Entailment, Causal Inference, Position

Conversational Causal Emotion Entailment (C2E2) is a task that aims at recognizing the causes corresponding to a target emotion in a conversation. The order of utterances in the conversation affects the causal inference. However, most current position encoding strategies ignore the order relation among utterances and speakers. To address the issue, we devise a novel position-aware graph to encode the entire conversation, fully modeling causal relations among utterances. The comprehensive experiments show that our method consistently achieves state-of-the-art performance on two challenging test sets, proving the effectiveness of our model. Our source code is available on Github: https://github.com/XiaojieGu/PAGE.
NCL: Textual Backdoor Defense Using Noise-augmented Contrastive Learning
Authors: Shengfang Zhai, Qingni Shen, Xiaoyi Chen, Weilong Wang, Cong Li, Yuejian Fang, Zhonghai Wu
Date: 2023-03-03 00:00:00
Tasks: backdoor defense, Contrastive Learning

At present, backdoor attacks attract attention as they do great harm to deep learning models. The adversary poisons the training data making the model being injected with a backdoor after being trained unconsciously by victims using the poisoned dataset. In the field of text, however, existing works do not provide sufficient defense against backdoor attacks. In this paper, we propose a Noise-augmented Contrastive Learning (NCL) framework to defend against textual backdoor attacks when training models with untrustworthy data. With the aim of mitigating the mapping between triggers and the target label, we add appropriate noise perturbing possible backdoor triggers, augment the training dataset, and then pull homology samples in the feature space utilizing contrastive learning objective. Experiments demonstrate the effectiveness of our method in defending three types of textual backdoor attacks, outperforming the prior works.
DWFormer: Dynamic Window transFormer for Speech Emotion Recognition
Authors: Shuaiqi Chen, Xiaofen Xing, Weibin Zhang, Weidong Chen, Xiangmin Xu
Date: 2023-03-03 00:00:00
Tasks: Emotion Recognition, Speech Emotion Recognition

Speech emotion recognition is crucial to human-computer interaction. The temporal regions that represent different emotions scatter in different parts of the speech locally. Moreover, the temporal scales of important information may vary over a large range within and across speech segments. Although transformer-based models have made progress in this field, the existing models could not precisely locate important regions at different temporal scales. To address the issue, we propose Dynamic Window transFormer (DWFormer), a new architecture that leverages temporal importance by dynamically splitting samples into windows. Self-attention mechanism is applied within windows for capturing temporal important information locally in a fine-grained way. Cross-window information interaction is also taken into account for global communication. DWFormer is evaluated on both the IEMOCAP and the MELD datasets. Experimental results show that the proposed model achieves better performance than the previous state-of-the-art methods.
Mixture of Soft Prompts for Controllable Data Generation
Authors: Derek Chen, Celine Lee, Yunan Lu, Domenic Rosati, Zhou Yu
Date: 2023-03-02 00:00:00
Tasks: Data Augmentation, Denoising, Few-Shot Learning, Prediction, Structured Prediction

Large language models (LLMs) effectively generate fluent text when the target output follows natural language patterns. However, structured prediction tasks confine the output format to a limited ontology, causing even very large models to struggle since they were never trained with such restrictions in mind. The difficulty of using LLMs for direct prediction is exacerbated in few-shot learning scenarios, which commonly arise due to domain shift and resource limitations. We flip the problem on its head by leveraging the LLM as a tool for data augmentation rather than direct prediction. Our proposed Mixture of Soft Prompts (MSP) serves as a parameter-efficient procedure for generating data in a controlled manner. Denoising mechanisms are further applied to improve the quality of synthesized data. Automatic metrics show our method is capable of producing diverse and natural text, while preserving label semantics. Moreover, MSP achieves state-of-the-art results on three benchmarks when compared against strong baselines. Our method offers an alternate data-centric approach for applying LLMs to complex prediction tasks.
Evolutionary Multi-Objective Algorithms for the Knapsack Problems with Stochastic Profits
Authors: Kokila Perera, Aneta Neumann, Frank Neumann
Date: 2023-03-03 00:00:00
Tasks: Combinatorial Optimization, Evolutionary Algorithms

Evolutionary multi-objective algorithms have been widely shown to be successful when utilized for a variety of stochastic combinatorial optimization problems. Chance constrained optimization plays an important role in complex real-world scenarios, as it allows decision makers to take into account the uncertainty of the environment. We consider a version of the knapsack problem with stochastic profits to guarantee a certain level of confidence in the profit of the solutions. We introduce the multi-objective formulations of the profit chance constrained knapsack problem and design three bi-objective fitness evaluation methods that work independently of the specific confidence level required. We evaluate our approaches using well-known multi-objective evolutionary algorithms GSEMO and NSGA-II. In addition, we introduce a filtering method for GSEMO that improves the quality of the final population by periodically removing certain solutions from the interim populations based on their confidence level. We show the effectiveness of our approaches on several benchmarks for both settings where the knapsack items have fixed uniform uncertainties and uncertainties that are positively correlated with the expected profit of an item.
Bayesian Posterior Perturbation Analysis with Integral Probability Metrics
Authors: Alfredo Garbuno-Inigo, Tapio Helin, Franca Hoffmann, Bamdad Hosseini
Date: 2023-03-02 00:00:00
Tasks: Bayesian Inference

In recent years, Bayesian inference in large-scale inverse problems found in science, engineering and machine learning has gained significant attention. This paper examines the robustness of the Bayesian approach by analyzing the stability of posterior measures in relation to perturbations in the likelihood potential and the prior measure. We present new stability results using a family of integral probability metrics (divergences) akin to dual problems that arise in optimal transport. Our results stand out from previous works in three directions: (1) We construct new families of integral probability metrics that are adapted to the problem at hand; (2) These new metrics allow us to study both likelihood and prior perturbations in a convenient way; and (3) our analysis accommodates likelihood potentials that are only locally Lipschitz, making them applicable to a wide range of nonlinear inverse problems. Our theoretical findings are further reinforced through specific and novel examples where the approximation rates of posterior measures are obtained for different types of perturbations and provide a path towards the convergence analysis of recently adapted machine learning techniques for Bayesian inverse problems such as data-driven priors and neural network surrogates.
Tuberculosis chest X-ray detection using CNN-based hybrid segmentation and classification approach
Authors: Iqbal, Ahmed; Usman, Muhammad; Ahmed, Zohair
Date: 2023-02-26 00:00:00
Proceeding: Biomedical Signal Processing and Control 2023 2

Tuberculosis still significantly impacts the world's population, with more than 10 million people getting sick each year. Researchers have focused on developing computer-aided diagnosis (CAD) technology based on X-ray imaging to support the identification of tuberculosis, and deep learning is a popular and efficient method. However, deep learning-based CAD approaches usually ignore the relationship between the two vision tasks of specific region segmentation and classification. In this research, we introduced a novel TB-UNet, which is based on dilated fusion block (DF) and Attention block (AB) block for accurate segmentation of lungs regions and achieved the highest results in terms of Precision (0.9574), Recall (0.9512), and F1score (0.8988), IoU (0.8168) and Accuracy (0.9770). We also proposed TB-DenseNet which is based on five dual convolution blocks, DenseNet-169 layer, and a feature fusion block for the precise classification of tuberculosis images. The experiments have been performed on three chest X-ray (CXR) datasets, segmented images, and original images are fed to TB-DenseNet for better classification. Furthermore, the proposed method is tested against simultaneously three different diseases, such as Pneumonia, COVID-19, and Tuberculous. The highest results are achieved in terms of Precision (0.9567), Recall (0.9510), F1score (0.9538), and Accuracy (0.9510). The achieved results reflect that our proposed method produces the highest accuracy compared to the state-of-the-art methods. The source code is available at: https://github.com/ahmedeqbal/TB-DenseNet.
Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle
Authors: Vanessa Mehlin, Sigurd Schacht, Carsten Lanquillon
Date: 2023-02-05 00:00:00
Tasks: Deep Learning

Deep Learning has enabled many advances in machine learning applications in the last few years. However, since current Deep Learning algorithms require much energy for computations, there are growing concerns about the associated environmental costs. Energy-efficient Deep Learning has received much attention from researchers and has already made much progress in the last couple of years. This paper aims to gather information about these advances from the literature and show how and at which points along the lifecycle of Deep Learning (IT-Infrastructure, Data, Modeling, Training, Deployment, Evaluation) it is possible to reduce energy consumption.
Next-generation Co-Packaged Optics for Future Disaggregated AI Systems
Authors: Sajjad Moazeni
Date: 2023-03-03 00:00:00
Tasks: Cloud Computing

Co-packaged optics is poised to solve the interconnect bandwidth bottleneck for GPUs and AI accelerators in near future. This technology can immediately boost today's AI/ML compute power to train larger neural networks that can perform more complex tasks. More importantly, co-packaged optics unlocks new system-level opportunities to rethink our conventional supercomputing and datacenter architectures. Disaggregation of memory and compute units is one of such new paradigms that can greatly speed up AI/ML workloads by providing low-latency and high-throughput performance, while maintaining flexibility to support conventional cloud computing applications as well. This paper gives a brief overview of state-of-the-art of co-packaged optical I/O and requirements of its next generations. We also discuss ideas to exploit co-packaged optics in disaggregated AI systems and possible future directions.
The Grossberg Code: Universal Neural Network Signatures of Perceptual Experience
Authors: Birgitta Dresp-Langley
Date: 2023-03-03 00:00:00

Two universal functional principles of Adaptive Resonance Theory simulate the brain code of all biological learning and adaptive intelligence. Low level representations of multisensory stimuli in their immediate environmental context are formed on the basis of bottom up activation and under the control of top down matching rules that integrate high level long term traces of contextual configuration. These universal coding principles lead to the establishment of lasting brain signatures of perceptual experience in all living species, from aplysiae to primates. They are revisited in this paper here on the basis of examples drawn from the original code and from some of the most recent related empirical findings on contextual modulation in the brain, highlighting the potential of Grossberg's pioneering insights and groundbreaking theoretical work for intelligent solutions in the domain of developmental and cognitive robotics.