pwc-archive/papers-with-abstracts huggingface dataset:
Conditional score-based diffusion models for Bayesian inference in infinite dimensions
Authors: Lorenzo Baldassari, Ali Siahkoohi, Josselin Garnier, Knut Solna, Maarten V. de Hoop
Date: 2023-05-28 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Bayesian Inference, Denoising

Since their initial introduction, score-based diffusion models (SDMs) have been successfully applied to solve a variety of linear inverse problems in finite-dimensional vector spaces due to their ability to efficiently approximate the posterior distribution. However, using SDMs for inverse problems in infinite-dimensional function spaces has only been addressed recently, primarily through methods that learn the unconditional score. While this approach is advantageous for some inverse problems, it is mostly heuristic and involves numerous computationally costly forward operator evaluations during posterior sampling. To address these limitations, we propose a theoretically grounded method for sampling from the posterior of infinite-dimensional Bayesian linear inverse problems based on amortized conditional SDMs. In particular, we prove that one of the most successful approaches for estimating the conditional score in finite dimensions - the conditional denoising estimator - can also be applied in infinite dimensions. A significant part of our analysis is dedicated to demonstrating that extending infinite-dimensional SDMs to the conditional setting requires careful consideration, as the conditional score typically blows up for small times, contrarily to the unconditional score. We conclude by presenting stylized and large-scale numerical examples that validate our approach, offer additional insights, and demonstrate that our method enables large-scale, discretization-invariant Bayesian inference.
Debias Coarsely, Sample Conditionally: Statistical Downscaling through Optimal Transport and Probabilistic Diffusion Models
Authors: Zhong Yi Wan, Ricardo Baptista, Yi-fan Chen, John Anderson, Anudhyan Boral, Fei Sha, Leonardo Zepeda-Núñez
Date: 2023-05-24 00:00:00
Proceeding: NeurIPS 2023 11

We introduce a two-stage probabilistic framework for statistical downscaling using unpaired data. Statistical downscaling seeks a probabilistic map to transform low-resolution data from a biased coarse-grained numerical scheme to high-resolution data that is consistent with a high-fidelity scheme. Our framework tackles the problem by composing two transformations: (i) a debiasing step via an optimal transport map, and (ii) an upsampling step achieved by a probabilistic diffusion model with a posteriori conditional sampling. This approach characterizes a conditional distribution without needing paired data, and faithfully recovers relevant physical statistics from biased samples. We demonstrate the utility of the proposed approach on one- and two-dimensional fluid flow problems, which are representative of the core difficulties present in numerical simulations of weather and climate. Our method produces realistic high-resolution outputs from low-resolution inputs, by upsampling resolutions of 8x and 16x. Moreover, our procedure correctly matches the statistics of physical quantities, even when the low-frequency content of the inputs and outputs do not match, a crucial but difficult-to-satisfy assumption needed by current state-of-the-art alternatives. Code for this work is available at: https://github.com/google-research/swirl-dynamics/tree/main/swirl_dynamics/projects/probabilistic_diffusion.
SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models
Authors: Martin Gonzalez, Nelson Fernandez, Thuy Tran, Elies Gherbi, Hatem Hajri, Nader Masmoudi
Date: 2023-05-23 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Image Generation

A potent class of generative models known as Diffusion Probabilistic Models (DPMs) has become prominent. A forward diffusion process adds gradually noise to data, while a model learns to gradually denoise. Sampling from pre-trained DPMs is obtained by solving differential equations (DE) defined by the learnt model, a process which has shown to be prohibitively slow. Numerous efforts on speeding-up this process have consisted on crafting powerful ODE solvers. Despite being quick, such solvers do not usually reach the optimal quality achieved by available slow SDE solvers. Our goal is to propose SDE solvers that reach optimal quality without requiring several hundreds or thousands of NFEs to achieve that goal. We propose Stochastic Explicit Exponential Derivative-free Solvers (SEEDS), improving and generalizing Exponential Integrator approaches to the stochastic case on several frameworks. After carefully analyzing the formulation of exact solutions of diffusion SDEs, we craft SEEDS to analytically compute the linear part of such solutions. Inspired by the Exponential Time-Differencing method, SEEDS use a novel treatment of the stochastic components of solutions, enabling the analytical computation of their variance, and contains high-order terms allowing to reach optimal quality sampling $\sim3$-$5\times$ faster than previous SDE methods. We validate our approach on several image generation benchmarks, showing that SEEDS outperform or are competitive with previous SDE solvers. Contrary to the latter, SEEDS are derivative and training free, and we fully prove strong convergence guarantees for them.
Discrete Diffusion Language Model for Long Text Summarization
Authors: Do Huu Dat, Do Duc Anh, Anh Tuan Luu, Wray Buntine
Date: 2024-06-25 00:00:00
Tasks: Abstractive Text Summarization, Decoder, Language Modeling, Language Modelling, Mamba, model, Text Generation, Text Summarization

While diffusion models excel at conditional generating high-quality images, prior works in discrete diffusion models were not evaluated on conditional long-text generation. In this work, we address the limitations of prior discrete diffusion models for conditional long-text generation, particularly in long sequence-to-sequence tasks such as abstractive summarization. Despite fast decoding speeds compared to autoregressive methods, previous diffusion models failed on the abstractive summarization task due to the incompatibility between the backbone architectures and the random noising process. To overcome these challenges, we introduce a novel semantic-aware noising process that enables Transformer backbones to handle long sequences effectively. Additionally, we propose CrossMamba, an adaptation of the Mamba model to the encoder-decoder paradigm, which integrates seamlessly with the random absorbing noising process. Our approaches achieve state-of-the-art performance on three benchmark summarization datasets: Gigaword, CNN/DailyMail, and Arxiv, outperforming existing discrete diffusion models on ROUGE metrics as well as possessing much faster speed in inference compared to autoregressive models.
LongLaMP: A Benchmark for Personalized Long-form Text Generation
Authors: Ishita Kumar, Snigdha Viswanathan, Sushrita Yerra, Alireza Salemi, Ryan A. Rossi, Franck Dernoncourt, Hanieh Deilamsalehy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, Chien Van Nguyen, Thien Huu Nguyen, Hamed Zamani
Date: 2024-06-27 00:00:00
Tasks: Form, Language Modelling, Text Generation

Long-text generation is seemingly ubiquitous in real-world applications of large language models such as generating an email or writing a review. Despite the fundamental importance and prevalence of long-text generation in many practical applications, existing work on personalized generation has focused on the generation of very short text. To overcome these limitations, we study the problem of personalized long-text generation, that is, generating long-text that is personalized for a specific user while being practically useful for the vast majority of real-world applications that naturally require the generation of longer text. In this work, we demonstrate the importance of user-specific personalization for long-text generation tasks and develop the Long-text Language Model Personalization (LongLaMP) Benchmark. LongLaMP provides a comprehensive and diverse evaluation framework for personalized long-text generation. Extensive experiments on LongLaMP for zero-shot and fine-tuned language tasks demonstrate the effectiveness of the proposed benchmark and its utility for developing and evaluating techniques for personalized long-text generation across a wide variety of long-text generation tasks. The results highlight the importance of personalization across a wide variety of long-text generation tasks. Finally, we release the benchmark for others to use for this important problem.
Adaptive Draft-Verification for Efficient Large Language Model Decoding
Authors: Xukun Liu, Bowen Lei, Ruqi Zhang, Dongkuan Xu
Date: 2024-06-27 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model

Large language model (LLM) decoding involves generating a sequence of tokens based on a given context, where each token is predicted one at a time using the model's learned probabilities. The typical autoregressive decoding method requires a separate forward pass through the model for each token generated, which is computationally inefficient and poses challenges for deploying LLMs in latency-sensitive scenarios. The main limitations of current decoding methods stem from their inefficiencies and resource demands. Existing approaches either necessitate fine-tuning smaller models, which is resource-intensive, or rely on fixed retrieval schemes to construct drafts for the next tokens, which lack adaptability and fail to generalize across different models and contexts. To address these issues, we introduce a novel methodology called ADED, which accelerates LLM decoding without requiring fine-tuning. Our approach involves an adaptive draft-verification process that evolves over time to improve efficiency. We utilize a tri-gram matrix-based LLM representation to dynamically approximate the output distribution of the LLM, allowing the model to adjust to changing token probabilities during the decoding process. Additionally, we implement a draft construction mechanism that effectively balances exploration and exploitation, ensuring that the drafts generated are both diverse and close to the true output distribution of the LLM. The importance of this design lies in its ability to optimize the draft distribution adaptively, leading to faster and more accurate decoding. Through extensive experiments on various benchmark datasets and LLM architectures, we demonstrate that ADED significantly accelerates the decoding process while maintaining high accuracy, making it suitable for deployment in a wide range of practical applications.
Spatial Focused Bitemporal Interactive Network for Remote Sensing Image Change Detection
Authors: Hang Sun, Yuan YAO, Lefei Zhang, Dong Ren
Date: 2024-07-08 00:00:00
Proceeding: IEEE Transactions on Geoscience and Remote Sensing 2024 7
Tasks: Change Detection, Diversity

Recently, transformers have been widely explored in remote sensing image change detection and achieved remarkable performance. However, most existing transformer-based change detection methods overlook exploring the spatiotemporal relationships between bitemporal images at the features within the same layer, which is crucial for learning discriminative features to perceive changes. Moreover, no explicit spatial constraint has been imposed on the final fused bitemporal features, leading to reduced detection performance on small targets. To address these issues, we propose a spatial focused bitemporal interactive network (SFBI-Net) for remote sensing image change detection. Specifically, a bitemporal spatiotemporal interactive (BSI) module is proposed, which performs global interactions on bitemporal features at the same network layer and supplements local information to obtain spatiotemporal relationships of bitemporal features for discriminative representation. Furthermore, a spatial focus diversity loss (SFD-Loss) is developed to maximize bitemporal features in the spatial dimension and further enhance the feature representation of change areas, especially small target areas. The experimental results on challenging benchmark datasets demonstrate the superiority of our SFBI-Net. The source code is available at https://github.com/Mryao-yuan/SFBI-Net.
Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification
Authors: J. Salamon, J. P. Bello
Date: 2017-01-23 00:00:00
Proceeding: IEEE Signal Processing Letters 2017 1
Tasks: Classification, Data Augmentation, Dictionary Learning, Environmental Sound Classification, Sound Classification

The ability of deep convolutional neural networks (CNNs) to learn discriminative spectro-temporal patterns makes them well suited to environmental sound classification. However, the relative scarcity of labeled data has impeded the exploitation of this family of high-capacity models. This study has two primary contributions: first, we propose a deep CNN architecture for environmental sound classification. Second, we propose the use of audio data augmentation for overcoming the problem of data scarcity and explore the influence of different augmentations on the performance of the proposed CNN architecture. Combined with data augmentation, the proposed model produces state-of-the-art results for environmental sound classification. We show that the improved performance stems from the combination of a deep, high-capacity model and an augmented training set: this combination outperforms both the proposed CNN without augmentation and a “shallow” dictionary learning model with augmentation. Finally, we examine the influence of each augmentation on the model's classification accuracy for each class, and observe that the accuracy for each class is influenced differently by each augmentation, suggesting that the performance of the model could be improved further by applying class-conditional data augmentation.
Multipeak Wavelength Detection of Spectrally Overlapped Fiber Bragg Grating Sensors Through a CNN-Based Autoencoder
Authors: Gabriel Rudloff, Marcelo A. Soto
Date: 2024-05-20 00:00:00
Proceeding: IEEE Sensors Journal 2024 5

This article presents a machine learning solution to identify the peak wavelengths of fiber Bragg grating (FBG) sensors multiplexed in a network with high spectral overlapping. Machine learning solutions generally require high-quality, extensive datasets, which can be difficult to obtain. In contrast, the proposed model corresponds to a convolutional neural network (CNN) autoencoder that can be trained in an unsupervised manner using only FBG spectrum data, without the need for information about the spectral positions of the sensors. The model is specifically designed to encode the spectral positions of overlapping FBG sensors as Dirac deltas, facilitating the straightforward extraction of spectral positions by computing their center of mass. The effectiveness and precision of the model are validated using both simulation and experimental data of a two-sensor serial array. In simulations, the model demonstrates promising adaptation capability, outperforming methods reported in the literature by over an order of magnitude in terms of mean absolute error (MAE). Meanwhile, when evaluated with experimental sensor data, the proposed autoencoder matches the performance of one of the most effective existing methods, but employing a much more efficient computing approach, thereby offering the potential for real-time inference of the spectral position of highly overlapping FBG sensors.
The Effect of Implicit Attitude on Self-Concept, Emotional Intelligence, Personality Characteristics and Ego Defense Styles
Authors: Mahmood Amirinia, Parisa Aghazadeh, Ali Baseri
Date: 2024-01-15 00:00:00
Proceeding: Psychology 2024 1
Tasks: Descriptive, Emotional Intelligence

We conducted this study to compare bank employees with high emotional intelligence to those with low emotional intelligence, assessing the impact of other implicit attitudes about one's self-concept on their self-concept, emotional intelligence, personality, and ego defense mechanisms. This study involved 153 participants. We classified the subjects into two experimental groups and one control group. We applied Beck's self-concept inventory, Bar-On emotional intelligence inventory, Defense Styles Questionnaire, and NEO Personality Inventory-Revised to assess the self-concept, emotional intelligence, ego defense styles, and personality characteristics of middle-aged participants from bank branches in Tehran, Iran. This study implemented control pre- and post-test designs. We analyzed the research data using descriptive analysis, partial correlation, ANCOVA, and MANCOVA at a significant level of 0.05. All experimental groups received Beck’s self-concept inventory. We analyzed the research data using descriptive analysis, partial correlation, and one-way multivariate analysis of covariance. In the experimental group, there was a significant drop in low emotional intelligence, a stronger negative correlation between self-concept and neuroticism, and a positive correlation with extraversion, conscientiousness, intrapersonal intelligence, general mood, and an immature defense style. However, we found no significant correlation between openness, agreeableness, interpersonal intelligence, adaptability, stress management, mature, and neurotic defense styles.
Ego Defensive Styles as Predictors of Emotional Intelligence and the Effects of Demographic Characteristics on Defensive Styles and Emotional Intelligence
Authors: Mahmood Amirinia, Parisa Aghazadeh &  Reyhaneh Bassam
Date: 2016-02-16 00:00:00
Proceeding: 3nd International Conference on Psychology Behavioral Sciences 2016 2
Tasks: Emotional Intelligence, Management

In this study, we first investigated the relationships between gender, age, and academic educational level of Iranian employees, as well as their ego defense styles and emotional intelligence, using a multivariate analysis of variance. Secondly, we examined the hypothesis that the ego defense styles (mature, neurotic, and immature) can predict the components of the Bar-On model of emotional intelligence with multiple regression models. The results showed that age and gender had a significant effect on ego defense styles, and age is a stress management facilitator. There were big differences between employees with different grade point averages on the ego defense styles questionnaire and the five main parts of the Bar-On model: general mood, stress management, adaptability, interpersonal, and total emotional intelligence scores.
Practical Implementation of KalmanNet for Accurate Data Fusion in Integrated Navigation
Authors: Jian Song, Wei Mei; Yunfeng Xu, Qiang Fu, Lina Bu
Date: 2024-07-09 00:00:00
Proceeding: IEEE Signal Processing Letters 2024 7
Tasks: Sensor Fusion, State Estimation

The extended Kalman filter has been widely used in sensor fusion to achieve integrated navigation and localization. Efficiently integrating multiple sensors requires prior knowledge about their errors for setting the filter. The recently emerged KalmanNet managed to use recurrent neural networks to learn prior knowledge from data and carry out state estimation for problems under non-linear dynamics with partial information. In this letter, the KalmanNet is implemented for integrated navigation using data from GPS/Wheels and the Inertial Measurement Unit. Therein, a practical strategy for the training algorithm of truncated backpropagation through time is presented by taking advantage of the first-order Markov property of the system state of the Kalman filter, which improves the training robustness and performance of the existing KalmanNet. Experimental results on the Michigan NCLT dataset show that our fusion KalmanNet significantly outperforms the conventional EKF-based fusion algorithm with an improvement of 20%∼40% in average RMSE.
Open-World Electrocardiogram Classification via Domain Knowledge-Driven Contrastive Learning
Authors: Shuang Zhou, Xiao Huang, Ninghao Liu, Wen Zhang, Yuan-Ting Zhang, Fu-Lai Chung
Date: 2024-07-17 00:00:00
Proceeding: Neural Networks 2024 7
Tasks: Classification, Clinical Knowledge, Contrastive Learning, Diagnostic, ECG Classification

Automatic electrocardiogram (ECG) classification provides valuable auxiliary information for assisting disease diagnosis and has received much attention in research. The success of existing classification models relies on fitting the labeled samples for every ECG type. However, in practice, well-annotated ECG datasets usually cover only limited ECG types. It thus raises an issue: conventional classification models trained with limited ECG types can only identify those ECG types that have already been observed in the training set, but fail to recognize unseen (or unknown) ECG types that exist in the wild and are not included in training data. In this work, we investigate an important problem called open-world ECG classification that can predict fine-grained observed ECG classes and identify unseen classes. Accordingly, we propose a customized method that first incorporates clinical knowledge into contrastive learning by generating ''hard negative'' samples to guide learning diagnostic ECG features (i.e., distinguishable representations), and then performs multi-hypersphere learning to learn compact ECG representations for classification. The experiment results on 12-lead ECG datasets (CPSC2018, PTB-XL, and Georgia) demonstrate that the proposed method outperforms the state-of-the-art methods. Specifically, our method achieves superior accuracy than the comparative methods on the unseen ECG class and certain seen classes. Overall, the investigated problem (i.e., open-world ECG classification) helps to draw attention to the reliability of automatic ECG diagnosis, and the proposed method is proven effective in tackling the challenges. The code and datasets will be open-sourced.
Detecting Side Effects of Adverse Drug Reactions Through Drug-Drug Interactions Using Graph Neural Networks and Self-Supervised Learning
Authors: OmKumar ChandraUmakantham, Srinitish Srinivasan, Varenya Pathak
Date: 2024-05-31 00:00:00
Proceeding: IEEE Access 2024 5
Tasks: Classification, Self-Supervised Learning, Type prediction

Adverse Drug Reactions(ADRs) due to drug-drug interactions present a public health problem worldwide that deserves attention due to its impact on mortality, morbidity, and healthcare costs. There have been major challenges in healthcare with the ever-increasing complexity of therapeutics and an aging population in many regions. At present, no standard method to detect such adverse drug reactions exists until otherwise reported by patients after the drug is released to the market. Further, several studies show that it is extremely challenging to detect these rare cases during clinical trials held before the drug is released. Therefore, a reliable and efficient technique to predict such side effects before the release of the drug to the market is the need of the hour. Through the power of Graph Neural Networks and the knowledge representation abilities of self-supervised learning, we designed an effective framework to model drug-drug interactions by leveraging the spatial and physical properties of drugs by representing them as molecular graphs. Through this approach, we developed a technique that resembles the dynamics of a chemical interaction. On training and testing this approach on the TwoSIDES Polypharmacy Dataset by Therapeutic Data Commons(TDC), we achieve a state of the art results by obtaining a precision of 75% and an accuracy of 90% on the test dataset. Further, we also perform a case study on the DrugBank dataset and compare our results on the interaction type prediction task to validate our approach on the drug-drug interaction domain and achieve excellent results with precision, F1, and accuracy of 99%. Our study and experimental approaches lay the groundwork for further research on side-effect prediction through drug-drug interaction and the use of Graph Neural Networks in the field of Molecular Biology.