pwc-archive/papers-with-abstracts huggingface dataset:
Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment
Authors: Shibbir Ahmed, Hongyang Gao, Hridesh Rajan
Date: 2024-01-26 00:00:00

Deep learning models are trained with certain assumptions about the data during the development stage and then used for prediction in the deployment stage. It is important to reason about the trustworthiness of the model's predictions with unseen data during deployment. Existing methods for specifying and verifying traditional software are insufficient for this task, as they cannot handle the complexity of DNN model architecture and expected outcomes. In this work, we propose a novel technique that uses rules derived from neural network computations to infer data preconditions for a DNN model to determine the trustworthiness of its predictions. Our approach, DeepInfer involves introducing a novel abstraction for a trained DNN model that enables weakest precondition reasoning using Dijkstra's Predicate Transformer Semantics. By deriving rules over the inductive type of neural network abstract representation, we can overcome the matrix dimensionality issues that arise from the backward non-linear computation from the output layer to the input layer. We utilize the weakest precondition computation using rules of each kind of activation function to compute layer-wise precondition from the given postcondition on the final output of a deep neural network. We extensively evaluated DeepInfer on 29 real-world DNN models using four different datasets collected from five different sources and demonstrated the utility, effectiveness, and performance improvement over closely related work. DeepInfer efficiently detects correct and incorrect predictions of high-accuracy models with high recall (0.98) and high F-1 score (0.84) and has significantly improved over prior technique, SelfChecker. The average runtime overhead of DeepInfer is low, 0.22 sec for all unseen datasets. We also compared runtime overhead using the same hardware settings and found that DeepInfer is 3.27 times faster than SelfChecker.
Cyclic Group Projection for Enumerating Quasi-Cyclic Codes Trapping Sets
Authors: Vasiliy Usatyuk, Yury Kuznetsov, Sergey Egorov
Date: 2024-01-26 00:00:00

This paper introduces a novel approach to enumerate and assess Trapping sets in quasi-cyclic codes, those with circulant sizes that are non-prime numbers. Leveraging the quasi-cyclic properties, the method employs a tabular technique to streamline the importance sampling step for estimating the pseudo-codeword weight of Trapping sets. The presented methodology draws on the mathematical framework established in the provided theorem, which elucidates the behavior of projection and lifting transformations on pseudo-codewords
Endowing Protein Language Models with Structural Knowledge
Authors: Dexiong Chen, Philip Hartout, Paolo Pellizzoni, Carlos Oliver, Karsten Borgwardt
Date: 2024-01-26 00:00:00
Tasks: Drug Design, Language Modeling, Language Modelling, Masked Language Modeling, Protein Function Prediction, Representation Learning

Understanding the relationships between protein sequence, structure and function is a long-standing biological challenge with manifold implications from drug design to our understanding of evolution. Recently, protein language models have emerged as the preferred method for this challenge, thanks to their ability to harness large sequence databases. Yet, their reliance on expansive sequence data and parameter sets limits their flexibility and practicality in real-world scenarios. Concurrently, the recent surge in computationally predicted protein structures unlocks new opportunities in protein representation learning. While promising, the computational burden carried by such complex data still hinders widely-adopted practical applications. To address these limitations, we introduce a novel framework that enhances protein language models by integrating protein structural data. Drawing from recent advances in graph transformers, our approach refines the self-attention mechanisms of pretrained language transformers by integrating structural information with structure extractor modules. This refined model, termed Protein Structure Transformer (PST), is further pretrained on a small protein structure database, using the same masked language modeling objective as traditional protein language models. Empirical evaluations of PST demonstrate its superior parameter efficiency relative to protein language models, despite being pretrained on a dataset comprising only 542K structures. Notably, PST consistently outperforms the state-of-the-art foundation model for protein sequences, ESM-2, setting a new benchmark in protein function prediction. Our findings underscore the potential of integrating structural information into protein language models, paving the way for more effective and efficient protein modeling Code and pretrained models are available at https://github.com/BorgwardtLab/PST.
Learning Local Control Barrier Functions for Hybrid Systems
Authors: Shuo Yang, Yu Chen, Xiang Yin, George J. Pappas, Rahul Mangharam
Date: 2024-01-26 00:00:00
Tasks: Autonomous Racing, Model Predictive Control

Hybrid dynamical systems are ubiquitous as practical robotic applications often involve both continuous states and discrete switchings. Safety is a primary concern for hybrid robotic systems. Existing safety-critical control approaches for hybrid systems are either computationally inefficient, detrimental to system performance, or limited to small-scale systems. To amend these drawbacks, in this paper, we propose a learning-enabled approach to construct local Control Barrier Functions (CBFs) to guarantee the safety of a wide class of nonlinear hybrid dynamical systems. The end result is a safe neural CBF-based switching controller. Our approach is computationally efficient, minimally invasive to any reference controller, and applicable to large-scale systems. We empirically evaluate our framework and demonstrate its efficacy and flexibility through two robotic examples including a high-dimensional autonomous racing case, against other CBF-based approaches and model predictive control.
Enhancement of a Text-Independent Speaker Verification System by using Feature Combination and Parallel-Structure Classifiers
Authors: Kerlos Atia Abdalmalak, Ascensión Gallardo-Antol'in
Date: 2024-01-26 00:00:00
Tasks: Speaker Verification, Text-Independent Speaker Verification

Speaker Verification (SV) systems involve mainly two individual stages: feature extraction and classification. In this paper, we explore these two modules with the aim of improving the performance of a speaker verification system under noisy conditions. On the one hand, the choice of the most appropriate acoustic features is a crucial factor for performing robust speaker verification. The acoustic parameters used in the proposed system are: Mel Frequency Cepstral Coefficients (MFCC), their first and second derivatives (Deltas and Delta- Deltas), Bark Frequency Cepstral Coefficients (BFCC), Perceptual Linear Predictive (PLP), and Relative Spectral Transform - Perceptual Linear Predictive (RASTA-PLP). In this paper, a complete comparison of different combinations of the previous features is discussed. On the other hand, the major weakness of a conventional Support Vector Machine (SVM) classifier is the use of generic traditional kernel functions to compute the distances among data points. However, the kernel function of an SVM has great influence on its performance. In this work, we propose the combination of two SVM-based classifiers with different kernel functions: Linear kernel and Gaussian Radial Basis Function (RBF) kernel with a Logistic Regression (LR) classifier. The combination is carried out by means of a parallel structure approach, in which different voting rules to take the final decision are considered. Results show that significant improvement in the performance of the SV system is achieved by using the combined features with the combined classifiers either with clean speech or in the presence of noise. Finally, to enhance the system more in noisy environments, the inclusion of the multiband noise removal technique as a preprocessing stage is proposed.
Combating Representation Learning Disparity with Geometric Harmonization
Authors: Zhihan Zhou, Jiangchao Yao, Feng Hong, Ya zhang, Bo Han, Yanfeng Wang
Date: 2023-10-26 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Representation Learning, Self-Supervised Learning

Self-supervised learning (SSL) as an effective paradigm of representation learning has achieved tremendous success on various curated datasets in diverse scenarios. Nevertheless, when facing the long-tailed distribution in real-world applications, it is still hard for existing methods to capture transferable and robust representation. Conventional SSL methods, pursuing sample-level uniformity, easily leads to representation learning disparity where head classes dominate the feature regime but tail classes passively collapse. To address this problem, we propose a novel Geometric Harmonization (GH) method to encourage category-level uniformity in representation learning, which is more benign to the minority and almost does not hurt the majority under long-tailed distribution. Specially, GH measures the population statistics of the embedding space on top of self-supervised learning, and then infer an fine-grained instance-wise calibration to constrain the space expansion of head classes and avoid the passive collapse of tail classes. Our proposal does not alter the setting of SSL and can be easily integrated into existing methods in a low-cost manner. Extensive results on a range of benchmark datasets show the effectiveness of GH with high tolerance to the distribution skewness. Our code is available at https://github.com/MediaBrain-SJTU/Geometric-Harmonization.
RGDAN: A random graph diffusion attention network for traffic prediction
Authors: Jin Fan, Wenchao Weng, Hao Tian, Huifeng Wu, Fu Zhu, Jia Wu
Date: 2024-01-16 00:00:00
Proceeding: journal 2024 1
Tasks: Traffic Prediction

Traffic Prediction based on graph structures is a challenging task given that road networks are typically complex structures and the data to be analyzed contains variable temporal features. Further, the quality of the spatial feature extraction is highly dependent on the weight settings of the graph structures. In the transportation field, the weights of these graph structures are currently calculated based on factors like the distance between roads. However, these methods do not take into account the characteristics of the road itself or the correlations between different traffic flows. Existing approaches usually pay more attention to local spatial dependencies extraction while global spatial dependencies are ignored. Another major problem is how to extract sufficient information at limited depth of graph structures. To address these challenges, we propose a Random Graph Diffusion Attention Network (RGDAN) for traffic prediction. RGDAN comprises a graph diffusion attention module and a temporal attention module. The graph diffusion attention module can adjust its weights by learning from data like a CNN to capture more realistic spatial dependencies. The temporal attention module captures the temporal correlations. Experiments on three large-scale public datasets demonstrate that RGDAN produces predictions with 2%–5% more precision than state-of-the-art methods.
iPIC-XAI: Improving PIC-XAI for Enhanced Image Captioning Explanation
Authors: Modafar Al-Shouha, Gábor Szűcs
Date: 2023-09-23 00:00:00
Proceeding: 2023 14th IEEE International Conference on Cognitive Infocommunications (CogInfoCom) 2023 9
Tasks: Image Captioning, TAG

Image captioning task with its complexity has taken advantage of the recent developments in Deep learning (DL). However, DL-models are fundamentally abstruse and explaining their behaviour is a challenge. In this paper we present an algorithm to explain an image captioning model behavior. We enhanced PIC-XAI performance by introducing various components; (1) we utilize CLIP multimodal similarity for more efficiency, (2) we consider the query dependency tag as a clue for elements with bigger size, (3) we provide an algorithm to automatically set the preprocessing blurring kernel size, and (4) we use a similarity comparison technique to get more relevant answers. Additionally, we provide an improved version of $XIC$ metric, aiming for more consistent objective evaluation for XAI methods in image captioning field.
Facial Beauty Analysis Using Distribution Prediction and CNN Ensembles
Authors: Ahmed Aman Ibrahim, Noah Hassan Ugail, Tazkia Hoodh Jayatileke, Millie Hope Saffery, Hassan Ugail
Date: 2023-12-10 00:00:00
Proceeding: International Conference on Software, Knowledge, Information Management and Applications (SKIMA) 2023 12
Tasks: Face Verification, Facial Beauty Prediction, General Classification, Prediction, Transfer Learning

Facial Beauty Prediction (FBP) is a computer vision task of quantifying the beauty of a face. Several solutions to this problem have benefitted immensely from the recent developments in deep learning. However, the majority of current methods train machine learning models to purely predict mean beauty scores, treating FBP solely as a regression task. In addition, deep learning based FBP approaches so far use transfer learning from models trained on general classification tasks such as ImageNet. We propose fine-tuning an ensemble of convolutional neural network (CNN) models originally trained on face verification tasks using a variety of loss functions such as Earth Mover's Distance (EMD) based loss. With this approach, our method can predict the entire beauty score distribution rather than just the mean, and the predicted mean scores have a higher Pearson Correlation (PC) compared to the ground truth scores. This method achieves state of the art results on the MEBeauty dataset in terms of mean absolute error, root mean squared error and PC between the predicted and the ground truth mean scores.
MEBeauty: a multi-ethnic facial beauty dataset in-the-wild
Authors: Irina Lebedeva, Yi Guo, Fangli Ying
Date: 2021-10-04 00:00:00
Proceeding: Emerging trends in Artificial Intelligence and Machine Learning 2021 10
Tasks: Face Recognition, Facial Beauty Prediction, Transfer Learning

Although facial beauty prediction (FBP) has achieved high accuracy on images captured in a constrained environment, it is still a challenging task on face images in-the-wild. Moreover, there is no FBP benchmark dataset that includes images with various ethnic, age, gender properties and unrestricted in terms of face expression and pose. In this work, the issue of FBP in real-life scenario is addressed and a multi-ethnic facial beauty dataset, namely MEBeauty, is introduced. All face images are captured in an unconstrained environment and rated by volunteers with various ethnicity, age and gender in order to avoid any cultural and social biases in beauty perception. Different well-known CNNs with layer-wise transfer learning are performed on the dataset. Moreover, the evaluation of knowledge learning from the face recognition task across FBP is conducted. The expected high number of aberrant and outlier faces is considered and the use of various robust loss functions in order to learn deep regression networks for facial beauty prediction is evaluated. Several FBP frameworks are performed on the proposed dataset and widely-used SCUT-FBP 5500 in order to compare their effectiveness on face images in constrained and unconstrained environments.
Def2Vec: Extensible Word Embeddings from Dictionary Definitions
Authors: Irene Morazzoni, Vincenzo Scotti, Roberto Tedesco
Date: 2023-12-16 00:00:00
Proceeding: ICNLSP 2023 12
Tasks: Chunking, named-entity-recognition, Named Entity Recognition, NER, Part-Of-Speech Tagging, POS, Semantic Similarity, Semantic Textual Similarity, Word Embeddings

Def2Vec introduces a novel paradigm for word embeddings, leveraging dictionary definitions to learn semantic representations. By constructing term-document matrices from definitions and applying Latent Semantic Analysis (LSA), Def2Vec generates embeddings that offer both strong performance and extensibility. In evaluations encompassing Part-of-Speech tagging, Named Entity Recognition, chunking, and semantic similarity, Def2Vec often matches or surpasses state-of-the-art models like Word2Vec, GloVe, and fastText. Our model’s second factorised matrix resulting from LSA enables efficient embedding extension for out-of-vocabulary words. By effectively reconciling the advantages of dictionary definitions with LSA-based embeddings, Def2Vec yields informative semantic representations, especially considering its reduced data requirements. This paper advances the understanding of word embedding generation by incorporating structured lexical information and efficient embedding extension.
On the similarity of Information Energy to Dark Energy
Authors: M. P. Gough, T. D. Carozzi, A. M. Buckley
Date: 2006-03-03 00:00:00

Information energy is shown here to have properties similar to those of dark energy. The energy associated with each information bit of the universe is found to be defined identically to the characteristic energy of a cosmological constant. Two independent methods are used to estimate the universe information content of ~10^91 bits, a value that provides an information energy total comparable to that of the dark energy. Information energy is also found to have a significantly negative equation of state parameter, w < -0.4, and thus exerts a negative pressure, similar to dark energy.
Wasserstein distributional robustness of neural networks
Authors: Xingjian Bai, Guangyi He, Yifan Jiang, Jan Obloj
Date: 2023-06-16 00:00:00
Proceeding: NeurIPS 2023 11

Deep neural networks are known to be vulnerable to adversarial attacks (AA). For an image recognition task, this means that a small perturbation of the original can result in the image being misclassified. Design of such attacks as well as methods of adversarial training against them are subject of intense research. We re-cast the problem using techniques of Wasserstein distributionally robust optimization (DRO) and obtain novel contributions leveraging recent insights from DRO sensitivity analysis. We consider a set of distributional threat models. Unlike the traditional pointwise attacks, which assume a uniform bound on perturbation of each input data point, distributional threat models allow attackers to perturb inputs in a non-uniform way. We link these more general attacks with questions of out-of-sample performance and Knightian uncertainty. To evaluate the distributional robustness of neural networks, we propose a first-order AA algorithm and its multi-step version. Our attack algorithms include Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) as special cases. Furthermore, we provide a new asymptotic estimate of the adversarial accuracy against distributional threat models. The bound is fast to compute and first-order accurate, offering new insights even for the pointwise AA. It also naturally yields out-of-sample performance guarantees. We conduct numerical experiments on the CIFAR-10 dataset using DNNs on RobustBench to illustrate our theoretical results. Our code is available at https://github.com/JanObloj/W-DRO-Adversarial-Methods.
MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries
Authors: Yixuan Tang, Yi Yang
Date: 2024-01-27 00:00:00
Tasks: Benchmarking, RAG, Retrieval, Retrieval-augmented Generation

Retrieval-augmented generation (RAG) augments large language models (LLM) by retrieving relevant knowledge, showing promising potential in mitigating LLM hallucinations and enhancing response quality, thereby facilitating the great adoption of LLMs in practice. However, we find that existing RAG systems are inadequate in answering multi-hop queries, which require retrieving and reasoning over multiple pieces of supporting evidence. Furthermore, to our knowledge, no existing RAG benchmarking dataset focuses on multi-hop queries. In this paper, we develop a novel dataset, MultiHop-RAG, which consists of a knowledge base, a large collection of multi-hop queries, their ground-truth answers, and the associated supporting evidence. We detail the procedure of building the dataset, utilizing an English news article dataset as the underlying RAG knowledge base. We demonstrate the benchmarking utility of MultiHop-RAG in two experiments. The first experiment compares different embedding models for retrieving evidence for multi-hop queries. In the second experiment, we examine the capabilities of various state-of-the-art LLMs, including GPT-4, PaLM, and Llama2-70B, in reasoning and answering multi-hop queries given the evidence. Both experiments reveal that existing RAG methods perform unsatisfactorily in retrieving and answering multi-hop queries. We hope MultiHop-RAG will be a valuable resource for the community in developing effective RAG systems, thereby facilitating greater adoption of LLMs in practice. The MultiHop-RAG and implemented RAG system is publicly available at https://github.com/yixuantt/MultiHop-RAG/.
AI prediction of cardiovascular events using opportunistic epicardial adipose tissue assessments from CT calcium score
Authors: Tao Hu, Joshua Freeze, Prerna Singh, Justin Kim, Yingnan Song, Hao Wu, Juhwan Lee, Sadeer Al-Kindi, Sanjay Rajagopalan, David L. Wilson, Ammar Hoori
Date: 2024-01-29 00:00:00
Tasks: Prediction

Background: Recent studies have used basic epicardial adipose tissue (EAT) assessments (e.g., volume and mean HU) to predict risk of atherosclerosis-related, major adverse cardiovascular events (MACE). Objectives: Create novel, hand-crafted EAT features, 'fat-omics', to capture the pathophysiology of EAT and improve MACE prediction. Methods: We segmented EAT using a previously-validated deep learning method with optional manual correction. We extracted 148 radiomic features (morphological, spatial, and intensity) and used Cox elastic-net for feature reduction and prediction of MACE. Results: Traditional fat features gave marginal prediction (EAT-volume/EAT-mean-HU/ BMI gave C-index 0.53/0.55/0.57, respectively). Significant improvement was obtained with 15 fat-omics features (C-index=0.69, test set). High-risk features included volume-of-voxels-having-elevated-HU-[-50, -30-HU] and HU-negative-skewness, both of which assess high HU, which as been implicated in fat inflammation. Other high-risk features include kurtosis-of-EAT-thickness, reflecting the heterogeneity of thicknesses, and EAT-volume-in-the-top-25%-of-the-heart, emphasizing adipose near the proximal coronary arteries. Kaplan-Meyer plots of Cox-identified, high- and low-risk patients were well separated with the median of the fat-omics risk, while high-risk group having HR 2.4 times that of the low-risk group (P<0.001). Conclusion: Preliminary findings indicate an opportunity to use more finely tuned, explainable assessments on EAT for improved cardiovascular risk prediction.