pwc-archive/papers-with-abstracts huggingface dataset:
Data-driven Conditional Instrumental Variables for Debiasing Recommender Systems
Authors: Zhirong Huang, Shichao Zhang, Debo Cheng, Jiuyong Li, Lin Liu, Guangquan Lu
Date: 2024-08-19 00:00:00
Tasks: Recommendation Systems, valid

In recommender systems, latent variables can cause user-item interaction data to deviate from true user preferences. This biased data is then used to train recommendation models, further amplifying the bias and ultimately compromising both recommendation accuracy and user satisfaction. Instrumental Variable (IV) methods are effective tools for addressing the confounding bias introduced by latent variables; however, identifying a valid IV is often challenging. To overcome this issue, we propose a novel data-driven conditional IV (CIV) debiasing method for recommender systems, called CIV4Rec. CIV4Rec automatically generates valid CIVs and their corresponding conditioning sets directly from interaction data, significantly reducing the complexity of IV selection while effectively mitigating the confounding bias caused by latent variables in recommender systems. Specifically, CIV4Rec leverages a variational autoencoder (VAE) to generate the representations of the CIV and its conditional set from interaction data, followed by the application of least squares to derive causal representations for click prediction. Extensive experiments on two real-world datasets, Movielens-10M and Douban-Movie, demonstrate that our CIV4Rec successfully identifies valid CIVs, effectively reduces bias, and consequently improves recommendation accuracy.
ExpoMamba: Exploiting Frequency SSM Blocks for Efficient and Effective Image Enhancement
Authors: Eashan Adhikarla, Kai Zhang, John Nicholson, Brian D. Davison
Date: 2024-08-19 00:00:00
Tasks: Computational Efficiency, Image Enhancement, Low-Light Image Enhancement

Low-light image enhancement remains a challenging task in computer vision, with existing state-of-the-art models often limited by hardware constraints and computational inefficiencies, particularly in handling high-resolution images. Recent foundation models, such as transformers and diffusion models, despite their efficacy in various domains, are limited in use on edge devices due to their computational complexity and slow inference times. We introduce ExpoMamba, a novel architecture that integrates components of the frequency state space within a modified U-Net, offering a blend of efficiency and effectiveness. This model is specifically optimized to address mixed exposure challenges, a common issue in low-light image enhancement, while ensuring computational efficiency. Our experiments demonstrate that ExpoMamba enhances low-light images up to 2-3x faster than traditional models with an inference time of 36.6 ms and achieves a PSNR improvement of approximately 15-20% over competing models, making it highly suitable for real-time image processing applications.
Deep Learning-based Machine Condition Diagnosis using Short-time Fourier Transformation Variants
Authors: Eduardo Jr Piedad, Zherish Galvin Mayordo, Eduardo Prieto-Araujo, Oriol Gomis-Bellmunt
Date: 2024-08-19 00:00:00

In motor condition diagnosis, electrical current signature serves as an alternative feature to vibration-based sensor data, which is a more expensive and invasive method. Machine learning (ML) techniques have been emerging in diagnosing motor conditions using only motor phase current signals. This study converts time-series motor current signals to time-frequency 2D plots using Short-time Fourier Transform (STFT) methods. The motor current signal dataset consists of 3,750 sample points with five classes - one healthy and four synthetically-applied motor fault conditions, and with five loading conditions: 0, 25, 50, 75, and 100%. Five transformation methods are used on the dataset: non-overlap and overlap STFTs, non-overlap and overlap realigned STFTs, and synchrosqueezed STFT. Then, deep learning (DL) models based on the previous Convolutional Neural Network (CNN) architecture are trained and validated from generated plots of each method. The DL models of overlap-STFT, overlap R-STFT, non-overlap STFT, non-overlap R-STFT, and synchrosqueezed-STFT performed exceptionally with an average accuracy of 97.65, 96.03, 96.08, 96.32, and 88.27%, respectively. Four methods outperformed the previous best ML method with 93.20% accuracy, while all five outperformed previous 2D-plot-based methods with accuracy of 80.25, 74.80, and 82.80%, respectively, using the same dataset, same DL architecture, and validation steps.
Debiased Contrastive Representation Learning for Mitigating Dual Biases in Recommender Systems
Authors: Zhirong Huang, Shichao Zhang, Debo Cheng, Jiuyong Li, Lin Liu, Guixian Zhang
Date: 2024-08-19 00:00:00
Tasks: Contrastive Learning, Diversity, Recommendation Systems, Representation Learning

In recommender systems, popularity and conformity biases undermine recommender effectiveness by disproportionately favouring popular items, leading to their over-representation in recommendation lists and causing an unbalanced distribution of user-item historical data. We construct a causal graph to address both biases and describe the abstract data generation mechanism. Then, we use it as a guide to develop a novel Debiased Contrastive Learning framework for Mitigating Dual Biases, called DCLMDB. In DCLMDB, both popularity bias and conformity bias are handled in the model training process by contrastive learning to ensure that user choices and recommended items are not unduly influenced by conformity and popularity. Extensive experiments on two real-world datasets, Movielens-10M and Netflix, show that DCLMDB can effectively reduce the dual biases, as well as significantly enhance the accuracy and diversity of recommendations.
Exploring Wavelet Transformations for Deep Learning-based Machine Condition Diagnosis
Authors: Eduardo Jr Piedad, Christian Ainsley Del Rosario, Eduardo Prieto-Araujo, Oriol Gomis-Bellmunt
Date: 2024-08-19 00:00:00
Tasks: Deep Learning

Deep learning (DL) strategies have recently been utilized to diagnose motor faults by simply analyzing motor phase current signals, offering a less costly and non-intrusive alternative to vibration sensors. This research transforms these time-series current signals into time-frequency 2D representations via Wavelet Transform (WT). The dataset for motor current signals includes 3,750 data points across five categories: one representing normal conditions and four representing artificially induced faults, each under five different load conditions: 0, 25, 50, 75, and 100%. The study employs five WT-based techniques: WT-Amor, WT-Bump, WT-Morse, WSST-Amor, and WSST-Bump. Subsequently, five DL models adopting prior Convolutional Neural Network (CNN) architecture were developed and tested using the transformed 2D plots from each method. The DL models for WT-Amor, WT-Bump, and WT-Morse showed remarkable effectiveness with peak model accuracy of 90.93, 89.20, and 93.73%, respectively, surpassing previous 2D-image-based methods that recorded accuracy of 80.25, 74.80, and 82.80% respectively using the identical dataset and validation protocol. Notably, the WT-Morse approach slightly exceeded the formerly highest ML technique, achieving a 93.20% accuracy. However, the two WSST methods that utilized synchrosqueezing techniques faced difficulty accurately classifying motor faults. The performance of Wavelet-based deep learning methods offers a compelling alternative for machine condition monitoring.
How to Make the Most of LLMs' Grammatical Knowledge for Acceptability Judgments
Authors: Yusuke Ide, Yuto Nishida, Miyu Oba, Yusuke Sakai, Justin Vasselli, Hidetaka Kamigaito, Taro Watanabe
Date: 2024-08-19 00:00:00

The grammatical knowledge of language models (LMs) is often measured using a benchmark of linguistic minimal pairs, where LMs are presented with a pair of acceptable and unacceptable sentences and required to judge which is acceptable. The existing dominant approach, however, naively calculates and compares the probabilities of paired sentences using LMs. Additionally, large language models (LLMs) have yet to be thoroughly examined in this field. We thus investigate how to make the most of LLMs' grammatical knowledge to comprehensively evaluate it. Through extensive experiments of nine judgment methods in English and Chinese, we demonstrate that a probability readout method, in-template LP, and a prompting-based method, Yes/No probability computing, achieve particularly high performance, surpassing the conventional approach. Our analysis reveals their different strengths, e.g., Yes/No probability computing is robust against token-length bias, suggesting that they harness different aspects of LLMs' grammatical knowledge. Consequently, we recommend using diverse judgment methods to evaluate LLMs comprehensively.
Meta-Learning on Augmented Gene Expression Profiles for Enhanced Lung Cancer Detection
Authors: Arya Hadizadeh Moghaddam, Mohsen Nayebi Kerdabadi, Cuncong Zhong, Zijun Yao
Date: 2024-08-19 00:00:00
Tasks: Meta-Learning, Transfer Learning

Gene expression profiles obtained through DNA microarray have proven successful in providing critical information for cancer detection classifiers. However, the limited number of samples in these datasets poses a challenge to employ complex methodologies such as deep neural networks for sophisticated analysis. To address this "small data" dilemma, Meta-Learning has been introduced as a solution to enhance the optimization of machine learning models by utilizing similar datasets, thereby facilitating a quicker adaptation to target datasets without the requirement of sufficient samples. In this study, we present a meta-learning-based approach for predicting lung cancer from gene expression profiles. We apply this framework to well-established deep learning methodologies and employ four distinct datasets for the meta-learning tasks, where one as the target dataset and the rest as source datasets. Our approach is evaluated against both traditional and deep learning methodologies, and the results show the superior performance of meta-learning on augmented source data compared to the baselines trained on single datasets. Moreover, we conduct the comparative analysis between meta-learning and transfer learning methodologies to highlight the efficiency of the proposed approach in addressing the challenges associated with limited sample sizes. Finally, we incorporate the explainability study to illustrate the distinctiveness of decisions made by meta-learning.
On the Foundations of Conflict-Driven Solving for Hybrid MKNF Knowledge Bases
Authors: Riley Kinahan, Spencer Killen, Kevin Wan, Jia-Huai You
Date: 2024-08-19 00:00:00

Hybrid MKNF Knowledge Bases (HMKNF-KBs) constitute a formalism for tightly integrated reasoning over closed-world rules and open-world ontologies. This approach allows for accurate modeling of real-world systems, which often rely on both categorical and normative reasoning. Conflict-driven solving is the leading approach for computationally hard problems, such as satisfiability (SAT) and answer set programming (ASP), in which MKNF is rooted. This paper investigates the theoretical underpinnings required for a conflict-driven solver of HMKNF-KBs. The approach defines a set of completion and loop formulas, whose satisfaction characterizes MKNF models. This forms the basis for a set of nogoods, which in turn can be used as the backbone for a conflict-driven solver.
Attention is a smoothed cubic spline
Authors: Zehua Lai, Lek-Heng Lim, Yucong Liu
Date: 2024-08-19 00:00:00
Tasks: Decoder

We highlight a perhaps important but hitherto unobserved insight: The attention module in a transformer is a smoothed cubic spline. Viewed in this manner, this mysterious but critical component of a transformer becomes a natural development of an old notion deeply entrenched in classical approximation theory. More precisely, we show that with ReLU-activation, attention, masked attention, encoder-decoder attention are all cubic splines. As every component in a transformer is constructed out of compositions of various attention modules (= cubic splines) and feed forward neural networks (= linear splines), all its components -- encoder, decoder, and encoder-decoder blocks; multilayered encoders and decoders; the transformer itself -- are cubic or higher-order splines. If we assume the Pierce-Birkhoff conjecture, then the converse also holds, i.e., every spline is a ReLU-activated encoder. Since a spline is generally just $C^2$, one way to obtain a smoothed $C^\infty$-version is by replacing ReLU with a smooth activation; and if this activation is chosen to be SoftMax, we recover the original transformer as proposed by Vaswani et al. This insight sheds light on the nature of the transformer by casting it entirely in terms of splines, one of the best known and thoroughly understood objects in applied mathematics.
Does Thought Require Sensory Grounding? From Pure Thinkers to Large Language Models
Authors: David J. Chalmers
Date: 2024-08-18 00:00:00
Tasks: Philosophy

Does the capacity to think require the capacity to sense? A lively debate on this topic runs throughout the history of philosophy and now animates discussions of artificial intelligence. I argue that in principle, there can be pure thinkers: thinkers that lack the capacity to sense altogether. I also argue for significant limitations in just what sort of thought is possible in the absence of the capacity to sense. Regarding AI, I do not argue directly that large language models can think or understand, but I rebut one important argument (the argument from sensory grounding) that they cannot. I also use recent results regarding language models to address the question of whether or how sensory grounding enhances cognitive capacities.
Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning
Authors: Tiansheng Huang, Gautam Bhattacharya, Pratik Joshi, Josh Kimball, Ling Liu
Date: 2024-08-18 00:00:00
Tasks: Philosophy, Safety Alignment

Safety aligned Large Language Models (LLMs) are vulnerable to harmful fine-tuning attacks \cite{qi2023fine}-- a few harmful data mixed in the fine-tuning dataset can break the LLMs's safety alignment. Existing mitigation strategies include alignment stage solutions \cite{huang2024vaccine, rosati2024representation} and fine-tuning stage solutions \cite{huang2024lazy,mukhoti2023fine}. However, our evaluation shows that both categories of defenses fail \textit{when some specific training hyper-parameters are chosen} -- a large learning rate or a large number of training epochs in the fine-tuning stage can easily invalidate the defense, which however, is necessary to guarantee finetune performance. To this end, we propose Antidote, a post-fine-tuning stage solution, which remains \textbf{\textit{agnostic to the training hyper-parameters in the fine-tuning stage}}. Antidote relies on the philosophy that by removing the harmful parameters, the harmful model can be recovered from the harmful behaviors, regardless of how those harmful parameters are formed in the fine-tuning stage. With this philosophy, we introduce a one-shot pruning stage after harmful fine-tuning to remove the harmful weights that are responsible for the generation of harmful content. Despite its embarrassing simplicity, empirical results show that Antidote can reduce harmful score while maintaining accuracy on downstream tasks.Our project page is at \url{https://huangtiansheng.github.io/Antidote_gh_page/}
Moonshine: Distilling Game Content Generators into Steerable Generative Models
Authors: Yuhe Nie, Michael Middleton, Tim Merino, Nidhushan Kanagaraja, Ashutosh Kumar, Zhan Zhuang, Julian Togelius
Date: 2024-08-18 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model

Procedural Content Generation via Machine Learning (PCGML) has enhanced game content creation, yet challenges in controllability and limited training data persist. This study addresses these issues by distilling a constructive PCG algorithm into a controllable PCGML model. We first generate a large amount of content with a constructive algorithm and label it using a Large Language Model (LLM). We use these synthetic labels to condition two PCGML models for content-specific generation, a diffusion model and the five-dollar model. This neural network distillation process ensures that the generation aligns with the original algorithm while introducing controllability through plain text. We define this text-conditioned PCGML as a Text-to-game-Map (T2M) task, offering an alternative to prevalent text-to-image multi-modal tasks. We compare our distilled models with the baseline constructive algorithm. Our analysis of the variety, accuracy, and quality of our generation demonstrates the efficacy of distilling constructive methods into controllable text-conditioned PCGML models.
SynTraC: A Synthetic Dataset for Traffic Signal Control from Traffic Monitoring Cameras
Authors: Tiejin Chen, Prithvi Shirke, Bharatesh Chakravarthi, Arpitsinh Vaghela, Longchao Da, Duo Lu, Yezhou Yang, Hua Wei
Date: 2024-08-18 00:00:00
Tasks: Management, Traffic Signal Control

This paper introduces SynTraC, the first public image-based traffic signal control dataset, aimed at bridging the gap between simulated environments and real-world traffic management challenges. Unlike traditional datasets for traffic signal control which aim to provide simplified feature vectors like vehicle counts from traffic simulators, SynTraC provides real-style images from the CARLA simulator with annotated features, along with traffic signal states. This image-based dataset comes with diverse real-world scenarios, including varying weather and times of day. Additionally, SynTraC also provides different reward values for advanced traffic signal control algorithms like reinforcement learning. Experiments with SynTraC demonstrate that it is still an open challenge to image-based traffic signal control methods compared with feature-based control methods, indicating our dataset can further guide the development of future algorithms. The code for this paper can be found in \url{https://github.com/DaRL-LibSignal/SynTraC}.SynTraC
Say My Name: a Model's Bias Discovery Framework
Authors: Massimiliano Ciranni, Luca Molinaro, Carlo Alberto Barbano, Attilio Fiandrotti, Vittorio Murino, Vito Paolo Pastore, Enzo Tartaglione
Date: 2024-08-18 00:00:00

In the last few years, due to the broad applicability of deep learning to downstream tasks and end-to-end training capabilities, increasingly more concerns about potential biases to specific, non-representative patterns have been raised. Many works focusing on unsupervised debiasing usually leverage the tendency of deep models to learn ``easier'' samples, for example by clustering the latent space to obtain bias pseudo-labels. However, the interpretation of such pseudo-labels is not trivial, especially for a non-expert end user, as it does not provide semantic information about the bias features. To address this issue, we introduce ``Say My Name'' (SaMyNa), the first tool to identify biases within deep models semantically. Unlike existing methods, our approach focuses on biases learned by the model. Our text-based pipeline enhances explainability and supports debiasing efforts: applicable during either training or post-hoc validation, our method can disentangle task-related information and proposes itself as a tool to analyze biases. Evaluation on traditional benchmarks demonstrates its effectiveness in detecting biases and even disclaiming them, showcasing its broad applicability for model diagnosis.
MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair
Authors: Meghdad Dehghan, Jie JW Wu, Fatemeh H. Fard, Ali Ouni
Date: 2024-08-18 00:00:00
Tasks: parameter-efficient fine-tuning, Program Repair

Large Language Models (LLMs) have shown high capabilities in several software development-related tasks such as program repair, documentation, code refactoring, debugging, and testing. However, training these models requires massive amount of data and significant computational resources. Adapters are specialized, small modules designed for parameter efficient fine-tuning of LLMs for specific tasks, domains, or applications without requiring extensive retraining of the entire model. These adapters offer a more efficient way to customize LLMs for particular needs, leveraging the pre-existing capabilities of the large model. Model (and adapter) merging have emerged as a technique to develop one model capable of multiple tasks, with minimal or no training required. Although model and adapter merging has shown promising performance in domains such as natural language processing and computer vision, its applicability to software engineering tasks remains underexplored. In this paper, we investigate the effectiveness of merged adapters within the context of software engineering, with a particular focus on the Automated Program Repair (APR) task, through our approach, MergeRepair. In particular, we merge multiple task-specific adapters using three different merging methods, including weight-averaging, ties, and dare-ties, and evaluate the performance of the merged adapter on the APR task. We introduce a continual merging approach, a novel method in which we sequentially merge the task-specific adapters where the order and weight of the merged adapters play a significant role. We further compare the performance of our approach with a baseline method consisting of equal-weight merging applied on parameters of different adapters, where all adapters are of equal importance.
Grammatical Error Feedback: An Implicit Evaluation Approach
Authors: Stefano Bannò, Kate Knill, Mark J. F. Gales
Date: 2024-08-18 00:00:00
Tasks: Grammatical Error Correction, Language Modeling, Language Modelling, Large Language Model

Grammatical feedback is crucial for consolidating second language (L2) learning. Most research in computer-assisted language learning has focused on feedback through grammatical error correction (GEC) systems, rather than examining more holistic feedback that may be more useful for learners. This holistic feedback will be referred to as grammatical error feedback (GEF). In this paper, we present a novel implicit evaluation approach to GEF that eliminates the need for manual feedback annotations. Our method adopts a grammatical lineup approach where the task is to pair feedback and essay representations from a set of possible alternatives. This matching process can be performed by appropriately prompting a large language model (LLM). An important aspect of this process, explored here, is the form of the lineup, i.e., the selection of foils. This paper exploits this framework to examine the quality and need for GEC to generate feedback, as well as the system used to generate feedback, using essays from the Cambridge Learner Corpus.