pwc-archive/papers-with-abstracts huggingface dataset:
Legilimens: Practical and Unified Content Moderation for Large Language Model Services
Authors: Jialin Wu, Jiangyi Deng, Shengyuan Pang, Yanjiao Chen, Jiayang Xu, Xinfeng Li, Wenyuan Xu
Date: 2024-08-28 00:00:00
Tasks: Data Augmentation, Language Modeling, Language Modelling, Large Language Model, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Given the societal impact of unsafe content generated by large language models (LLMs), ensuring that LLM services comply with safety standards is a crucial concern for LLM service providers. Common content moderation methods are limited by an effectiveness-and-efficiency dilemma, where simple models are fragile while sophisticated models consume excessive computational resources. In this paper, we reveal for the first time that effective and efficient content moderation can be achieved by extracting conceptual features from chat-oriented LLMs, despite their initial fine-tuning for conversation rather than content moderation. We propose a practical and unified content moderation framework for LLM services, named Legilimens, which features both effectiveness and efficiency. Our red-team model-based data augmentation enhances the robustness of Legilimens against state-of-the-art jailbreaking. Additionally, we develop a framework to theoretically analyze the cost-effectiveness of Legilimens compared to other methods. We have conducted extensive experiments on five host LLMs, seventeen datasets, and nine jailbreaking methods to verify the effectiveness, efficiency, and robustness of Legilimens against normal and adaptive adversaries. A comparison of Legilimens with both commercial and academic baselines demonstrates the superior performance of Legilimens. Furthermore, we confirm that Legilimens can be applied to few-shot scenarios and extended to multi-label classification tasks.
Awes, Laws, and Flaws From Today's LLM Research
Authors: Adrian de Wynter
Date: 2024-08-27 00:00:00
Tasks: Ethics, Language Modeling, Language Modelling, Large Language Model

We perform a critical examination of the scientific methodology behind contemporary large language model (LLM) research. For this we assess over 2,000 research works based on criteria typical of what is considered good research (e.g. presence of statistical tests and reproducibility) and cross-validate it with arguments that are at the centre of controversy (e.g., claims of emergent behaviour, the use of LLMs as evaluators). We find multiple trends, such as declines in claims of emergent behaviour and ethics disclaimers; the rise of LLMs as evaluators in spite of a lack of consensus from the community about their useability; and an increase of claims of LLM reasoning abilities, typically without leveraging human evaluation. This paper underscores the need for more scrutiny and rigour by and from this field to live up to the fundamentals of a responsible scientific method that is ethical, reproducible, systematic, and open to criticism.
DualKanbaFormer: An Efficient Selective Sparse Framework for Multimodal Aspect-based Sentiment Analysis
Authors: Adamu Lawan, Juhua Pu, Haruna Yunusa, Muhammad Lawan, Aliyu Umar, Adamu Sani Yahya, Mahmoud Basi
Date: 2024-08-27 00:00:00
Tasks: Aspect-Based Sentiment Analysis, Kolmogorov-Arnold Networks, Mamba, Sentiment Analysis

Multimodal Aspect-based Sentiment Analysis (MABSA) enhances sentiment detection by integrating textual data with complementary modalities, such as images, to provide a more refined and comprehensive understanding of sentiment. However, conventional attention mechanisms, despite notable benchmarks, are hindered by quadratic complexity, limiting their ability to fully capture global contextual dependencies and rich semantic information in both modalities. To address this limitation, we introduce DualKanbaFormer, a novel framework that leverages parallel Textual and Visual KanbaFormer modules for robust multimodal analysis. Our approach incorporates Aspect-Driven Sparse Attention (ADSA) to dynamically balance coarse-grained aggregation and fine-grained selection for aspect-focused precision, ensuring the preservation of both global context awareness and local precision in textual and visual representations. Additionally, we utilize the Selective State Space Model (Mamba) to capture extensive global semantic information across both modalities. Furthermore, We replace traditional feed-forward networks and normalization with Kolmogorov-Arnold Networks (KANs) and Dynamic Tanh (DyT) to enhance non-linear expressivity and inference stability. To facilitate the effective integration of textual and visual features, we design a multimodal gated fusion layer that dynamically optimizes inter-modality interactions, significantly enhancing the models efficacy in MABSA tasks. Comprehensive experiments on two publicly available datasets reveal that DualKanbaFormer consistently outperforms several state-of-the-art (SOTA) models.
Pitfalls and Outlooks in Using COMET
Authors: Vilém Zouhar, Pinzhen Chen, Tsz Kin Lam, Nikita Moghe, Barry Haddow
Date: 2024-08-27 00:00:00
Tasks: Machine Translation, Translation

The COMET metric has blazed a trail in the machine translation community, given its strong correlation with human judgements of translation quality. Its success stems from being a modified pre-trained multilingual model finetuned for quality assessment. However, it being a machine learning model also gives rise to a new set of pitfalls that may not be widely known. We investigate these unexpected behaviours from three aspects: 1) technical: obsolete software versions and compute precision; 2) data: empty content, language mismatch, and translationese at test time as well as distribution and domain biases in training; 3) usage and reporting: multi-reference support and model referencing in the literature. All of these problems imply that COMET scores are not comparable between papers or even technical setups and we put forward our perspective on fixing each issue. Furthermore, we release the sacreCOMET package that can generate a signature for the software and model configuration as well as an appropriate citation. The goal of this work is to help the community make more sound use of the COMET metric.
PDSR: A Privacy-Preserving Diversified Service Recommendation Method on Distributed Data
Authors: Lina Wang, Huan Yang, Yiran Shen, Chao Liu, Lianyong Qi, Xiuzhen Cheng, Feng Li
Date: 2024-08-28 00:00:00
Tasks: Collaborative Filtering, Diversity, Privacy Preserving

The last decade has witnessed a tremendous growth of service computing, while efficient service recommendation methods are desired to recommend high-quality services to users. It is well known that collaborative filtering is one of the most popular methods for service recommendation based on QoS, and many existing proposals focus on improving recommendation accuracy, i.e., recommending high-quality redundant services. Nevertheless, users may have different requirements on QoS, and hence diversified recommendation has been attracting increasing attention in recent years to fulfill users' diverse demands and to explore potential services. Unfortunately, the recommendation performances relies on a large volume of data (e.g., QoS data), whereas the data may be distributed across multiple platforms. Therefore, to enable data sharing across the different platforms for diversified service recommendation, we propose a Privacy-preserving Diversified Service Recommendation (PDSR) method. Specifically, we innovate in leveraging the Locality-Sensitive Hashing (LSH) mechanism such that privacy-preserved data sharing across different platforms is enabled to construct a service similarity graph. Based on the similarity graph, we propose a novel accuracy-diversity metric and design a $2$-approximation algorithm to select $K$ services to recommend by maximizing the accuracy-diversity measure. Extensive experiments on real datasets are conducted to verify the efficacy of our PDSR method.
Lyrically Speaking: Exploring the Link Between Lyrical Emotions, Themes and Depression Risk
Authors: Pavani Chowdary, Bhavyajeet Singh, Rajat Agarwal, Vinoo Alluri
Date: 2024-08-28 00:00:00
Tasks: Recommendation Systems

Lyrics play a crucial role in affecting and reinforcing emotional states by providing meaning and emotional connotations that interact with the acoustic properties of the music. Specific lyrical themes and emotions may intensify existing negative states in listeners and may lead to undesirable outcomes, especially in listeners with mood disorders such as depression. Hence, it is important for such individuals to be mindful of their listening strategies. In this study, we examine online music consumption of individuals at risk of depression in light of lyrical themes and emotions. Lyrics obtained from the listening histories of 541 Last.fm users, divided into At-Risk and No-Risk based on their mental well-being scores, were analyzed using natural language processing techniques. Statistical analyses of the results revealed that individuals at risk for depression prefer songs with lyrics associated with low valence and low arousal. Additionally, lyrics associated with themes of denial, self-reference, and ambivalence were preferred. In contrast, themes such as liberation, familiarity, and activity are not as favored. This study opens up the possibility of an approach to assessing depression risk from the digital footprint of individuals and potentially developing personalized recommendation systems.
Accelerating Image-based Pest Detection on a Heterogeneous Multi-core Microcontroller
Authors: Luca Bompani, Luca Crupi, Daniele Palossi, Olmo Baldoni, Davide Brunelli, Francesco Conti, Manuele Rusci, Luca Benini
Date: 2024-08-28 00:00:00

The codling moth pest poses a significant threat to global crop production, with potential losses of up to 80% in apple orchards. Special camera-based sensor nodes are deployed in the field to record and transmit images of trapped insects to monitor the presence of the pest. This paper investigates the embedding of computer vision algorithms in the sensor node using a novel State-of-the-Art Microcontroller Unit (MCU), the GreenWaves Technologies' GAP9 System-on-Chip, which combines 10 RISC-V general purposes cores with a convolution hardware accelerator. We compare the performance of a lightweight Viola-Jones detector algorithm with a Convolutional Neural Network (CNN), MobileNetV3-SSDLite, trained for the pest detection task. On two datasets that differentiate for the distance between the camera sensor and the pest targets, the CNN generalizes better than the other method and achieves a detection accuracy between 83% and 72%. Thanks to the GAP9's CNN accelerator, the CNN inference task takes only 147 ms to process a 320$\times$240 image. Compared to the GAP8 MCU, which only relies on general-purpose cores for processing, we achieved 9.5$\times$ faster inference speed. When running on a 1000 mAh battery at 3.7 V, the estimated lifetime is approximately 199 days, processing an image every 30 seconds. Our study demonstrates that the novel heterogeneous MCU can perform end-to-end CNN inference with an energy consumption of just 4.85 mJ, matching the efficiency of the simpler Viola-Jones algorithm and offering power consumption up to 15$\times$ lower than previous methods. Code at: https://github.com/Bomps4/TAFE_Pest_Detection
Enhancing Biomedical Relation Extraction with Transformer Models using Shortest Dependency Path Features and Triplet Information
Authors: Vani Kanjirangat, Fabio Rinaldi
Date: 2021-10-01 00:00:00
Proceeding: Journal of Biomedical Informatics 2021 10
Tasks: Relation, Relation Extraction, Sentence, Sentence-Pair Classification, Triplet

Entity relation extraction plays an important role in the biomedical, healthcare, and clinical research areas. Recently, pre-trained models based on transformer architectures and their variants have shown remarkable performances in various natural language processing tasks. Most of these variants were based on slight modifications in the architectural components, representation schemes and augmenting data using distant supervision methods. In distantly supervised methods, one of the main challenges is pruning out noisy samples. A similar situation can arise when the training samples are not directly available but need to be constructed from the given dataset. The BioCreative V Chemical Disease Relation (CDR) task provides a dataset that does not explicitly offer mention-level gold annotations and hence replicates the above scenario. Selecting the representative sentences from the given abstract or document text that could convey a potential entity relationship becomes essential. Most of the existing methods in literature propose to either consider the entire text or all the sentences which contain the entity mentions. This could be a computationally expensive and time consuming approach. This paper presents a novel approach to handle such scenarios, specifically in biomedical relation extraction. We propose utilizing the Shortest Dependency Path (SDP) features for constructing data samples by pruning out noisy information and selecting the most representative samples for model learning. We also utilize triplet information in model learning using the biomedical variant of BERT, viz., BioBERT. The problem is represented as a sentence pair classification task using the sentence and the entity-relation pair as input. We analyze the approach on both intra-sentential and inter-sentential relations in the CDR dataset. The proposed approach that utilizes the SDP and triplet features presents promising results, specifically on the inter-sentential relation extraction task. We make the code used for this work publicly available on Github.
Deep learning with RGB and thermal images onboard a drone for monitoring operations
Authors: Simon Speth, Artur Gonçalves, Bastien Rigault, Satoshi Suzuki, Mondher Bouazizi, Yutaka Matsuo, Helmut Prendinger
Date: 2022-05-31 00:00:00
Proceeding: Journal of Field Robotics 2022 5
Tasks: Human Detection, Multispectral Object Detection, object-detection, Object Detection, Semantic Segmentation

This article describes the artificial intelligence (AI) component of a drone for monitoring and patrolling tasks associated with disaster relief missions in specific restricted disaster scenarios, as specified by the Advanced Robotics Foundation in Japan. The AI component uses deep learning models for environment recognition and object detection. For environment recognition, we use semantic segmentation, or pixel-wise labeling, based on RGB images. Object detection is key for detecting and locating people in need. Since people are relatively small objects from the drone perspective, we use both RGB and thermal images. To train our models, we created a novel multispectral and publicly available data set of people. We used a geo-location method to locate people on the ground. The semantic segmentation models were extensively tested using different feature extractors. We created two dedicated data sets, which we have made publicly available. Compared with the baseline model, the best-performing model could increase the mean intersection over union (IoU) by 1.3%. Furthermore, we compared two types of person detection models. The first one is an ensemble model that combines RGB and thermal information via “late fusion”; the second one is a 4-channel model that combines these two types of information in an “early fusion” manner. The results suggest that the 4-channel model had a 40.6% increase of average precision for stricter IoU values (0.75) compared with the ensemble model and a 5.8% increase in the average precision compared with the thermal model. All models were deployed and tested on the NVIDIA AGX Xavier platform. To the best of our knowledge, this study was the first to use both RGB and thermal data from the perspective of a drone for monitoring tasks.
Transformer-Based Multimodal Fusion for Early Diagnosis of Alzheimer's Disease Using Structural MRI And PET
Authors: Yuanwang Zhang, Kaicong Sun, Yuxiao Liu, Dinggang Shen
Date: 2023-04-18 00:00:00
Proceeding: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI) 2023 4

Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) are both the widely used imaging modalities for early diagnosis of Alzheimer's disease (AD). Combining these two modalities allow using both anatomical and metabolic information for evaluating brain status. However, the commonly-used multimodal fusion strategy, i.e., through channel concatenation, cannot effectively exploit complementary information among these two modalities. To encourage effective information exchange between structural MRI (sMRI) and FDG-PET as used in our study for early AD diagnosis, we propose a novel transformer-based multimodal fusion framework. Specially, our proposed model composes of three parts: 1) Feature extraction based on adversarial training; 2) Feature fusion based on multimodal transformer through cross-attention mechanism; 3) Classification head based on full connection. By resorting to adversarial learning, the feature gap between two modalities becomes smaller, thus easing the cross-attention operation to achieve more effective fusion. In the experiment, we show that our model outperforms other representative models by a large margin.
Vector Quantized Multi-modal Guidance for Alzheimer’s Disease Diagnosis Based on Feature Imputation
Authors: Yuanwang Zhang, Kaicong Sun, Yuxiao Liu, Zaixin Ou, Dinggang Shen
Date: 2023-10-08 00:00:00
Proceeding: International Workshop on Machine Learning in Medical Imaging 2023 10
Tasks: Imputation, Quantization

Magnetic Resonance Imaging (MRI) and positron emission tomography (PET) are the most used imaging modalities for Alzheimer’s disease (AD) diagnosis in clinics. Although PET can better capture AD-specific pathologies than MRI, it is less used compared with MRI due to high cost and radiation exposure. Imputing PET images from MRI is one way to bypass the issue of unavailable PET, but is challenging due to severe ill-posedness. Instead, we propose to directly impute classification-oriented PET features and combine them with real MRI to improve the overall performance of AD diagnosis. In order to more effectively impute PET features, we discretize the feature space by vector quantization and employ transformer to perform feature transition between MRI and PET. Our model is composed of three stages including codebook generation, mapping construction, and classifier enhancement based on combined features. We employ paired MRI-PET data during training to enhance the performance of MRI data during inference. Experimental results on ADNI dataset including 1346 subjects show a boost in classification performance of MRI without requiring PET. Our proposed method also outperforms other state-of-the-art data imputation methods.
Multiview Contrastive Learning for Unsupervised Domain Adaptation in Brain–Computer Interfaces
Authors: Sepehr Asgarian, Ze Wang, Feng Wan, Chi Man Wong, Feng Liu, Yalda Mohsenzadeh, Boyu Wang, Charles X. Ling
Date: 2024-02-15 00:00:00
Proceeding: IEEE Transactions on Instrumentation and Measurement 2024 2
Tasks: Classification, Contrastive Learning, Domain Adaptation, EEG, Motor Imagery, Transfer Learning, Unsupervised Domain Adaptation

Domain adaptation has gained significant attention to address the nonstationarity problem in electroencephalography (EEG) data in motor imagery (MI) classification. In MI classification, domain adaptation addresses cross-session variations, enhancing the classifier’s generalization capabilities. However, the existing methods have struggled to effectively capture both temporal and spatial features, resulting in limited classification accuracy. To tackle this issue, we propose the multiview adversarial contrastive network (MACNet). The proposed MACNet simultaneously learns spatial and temporal features in two different views: Euclidean and Riemannian. Furthermore, we introduce a multilevel domain mix-up technique to enhance domain alignment at both signal and embedding levels. The proposed MACNet method is evaluated on three public datasets. It achieves an accuracy of 83.79% on the BCI Competition IV dataset, 80.00% on the open source brain-machine interface (OpenBMI) dataset, and 85.83% on the sensorimotor rthythms (SMR) dataset that outperforms previous methods in cross-session transfer learning.
IBO: Inpainting-Based Occlusion to Enhance Explainable Artificial Intelligence Evaluation in Histopathology
Authors: Pardis Afshar, Sajjad Hashembeiki, Pouya Khani, Emad Fatemizadeh, Mohammad Hossein Rohban
Date: 2024-08-29 00:00:00
Tasks: Classification, Denoising, Explainable artificial intelligence, Explainable Artificial Intelligence (XAI), Interpretability Techniques for Deep Learning

Histopathological image analysis is crucial for accurate cancer diagnosis and treatment planning. While deep learning models, especially convolutional neural networks, have advanced this field, their "black-box" nature raises concerns about interpretability and trustworthiness. Explainable Artificial Intelligence (XAI) techniques aim to address these concerns, but evaluating their effectiveness remains challenging. A significant issue with current occlusion-based XAI methods is that they often generate Out-of-Distribution (OoD) samples, leading to inaccurate evaluations. In this paper, we introduce Inpainting-Based Occlusion (IBO), a novel occlusion strategy that utilizes a Denoising Diffusion Probabilistic Model to inpaint occluded regions in histopathological images. By replacing cancerous areas with realistic, non-cancerous tissue, IBO minimizes OoD artifacts and preserves data integrity. We evaluate our method on the CAMELYON16 dataset through two phases: first, by assessing perceptual similarity using the Learned Perceptual Image Patch Similarity (LPIPS) metric, and second, by quantifying the impact on model predictions through Area Under the Curve (AUC) analysis. Our results demonstrate that IBO significantly improves perceptual fidelity, achieving nearly twice the improvement in LPIPS scores compared to the best existing occlusion strategy. Additionally, IBO increased the precision of XAI performance prediction from 42% to 71% compared to traditional methods. These results demonstrate IBO's potential to provide more reliable evaluations of XAI techniques, benefiting histopathology and other applications. The source code for this study is available at https://github.com/a-fsh-r/IBO.
FEUNet: a flexible and effective U-shaped network for image denoising
Authors: Wencong Wu, Guannan Lv, Shicheng Liao, Yungang Zhang
Date: 2023-01-16 00:00:00
Proceeding: Signal, Image and Video Processing 2023 1
Tasks: Denoising, Image Denoising

Over the recent years, deep convolutional neural networks based models have been absolutely attractive in image denoising field due to their favorable performance. However, many existing deep neural network based image denoising models lack flexibility for spatially variant or real-world noise, which restricts the application of these models in real denoising scenes. In this paper, we propose a flexible and effective U-shaped network (FEUNet), which is effective in a wide range of noise levels, and can deal with spatially variant noise. The adjustable noise level map is used as the input of the FEUNet to enhance its flexibility. The U-Net is utilized to enhance the effectiveness of the proposed model. Experimental results have verified that the proposed FEUNet can obtain competitive denoising performances on many denoising tasks compared with the state-of-the-art denoising methods, which makes the proposed FEUNet well suited for the practical image denoising tasks.