pwc-archive/papers-with-abstracts huggingface dataset:
Low-Power Encoding for PAM-3 DRAM Bus
Authors: Jonghyeon Nam, Jaeduk Han, Hokeun Kim
Date: 2024-10-16 00:00:00

The 3-level pulse amplitude modulation (PAM-3) signaling is expected to be widely used in memory interfaces for its greater voltage margins compared to PAM-4. To maximize the benefit of PAM-3, we propose three low-power data encoding algorithms: PAM3-DBI, PAM3-MF, and PAM3-SORT. With the DRAM memory traces from the gem5 computer architecture simulator running benchmarks, we evaluate the energy efficiency of our three PAM-3 encoding techniques. The experimental results show the proposed algorithms can reduce termination power for high-speed memory links significantly by 41% to 90% for benchmark programs.
Online Energy Optimization in GPUs: A Multi-Armed Bandit Approach
Authors: Xiongxiao Xu, Solomon Abera Bekele, Brice Videau, Kai Shu
Date: 2024-10-03 00:00:00
Tasks: energy management, GPU

Energy consumption has become a critical design metric and a limiting factor in the development of future computing architectures, from small wearable devices to large-scale leadership computing facilities. The predominant methods in energy management optimization are focused on CPUs. However, GPUs are increasingly significant and account for the majority of energy consumption in heterogeneous high performance computing (HPC) systems. Moreover, they typically rely on either purely offline training or a hybrid of offline and online training, which are impractical and lead to energy loss during data collection. Therefore, this paper studies a novel and practical online energy optimization problem for GPUs in HPC scenarios. The problem is challenging due to the inherent performance-energy trade-offs of GPUs, the exploration & exploitation dilemma across frequencies, and the lack of explicit performance counters in GPUs. To address these challenges, we formulate the online energy consumption optimization problem as a multi-armed bandit framework and develop a novel bandit based framework EnergyUCB. EnergyUCB is designed to dynamically adjust GPU core frequencies in real-time, reducing energy consumption with minimal impact on performance. Specifically, the proposed framework EnergyUCB (1) balances the performance-energy trade-off in the reward function, (2) effectively navigates the exploration & exploitation dilemma when adjusting GPU core frequencies online, and (3) leverages the ratio of GPU core utilization to uncore utilization as a real-time GPU performance metric. Experiments on a wide range of real-world HPC benchmarks demonstrate that EnergyUCB can achieve substantial energy savings. The code of EnergyUCB is available at https://github.com/XiongxiaoXu/EnergyUCB-Bandit.
Enterprise Benchmarks for Large Language Model Evaluation
Authors: Bing Zhang, Mikio Takeuchi, Ryo Kawahara, Shubhi Asthana, Md. Maruf Hossain, Guang-jie Ren, Kate Soule, Yada Zhu
Date: 2024-10-11 00:00:00
Tasks: Benchmarking, Language Model Evaluation, Language Modeling, Language Modelling, Large Language Model, model

The advancement of large language models (LLMs) has led to a greater challenge of having a rigorous and systematic evaluation of complex tasks performed, especially in enterprise applications. Therefore, LLMs need to be able to benchmark enterprise datasets for various tasks. This work presents a systematic exploration of benchmarking strategies tailored to LLM evaluation, focusing on the utilization of domain-specific datasets and consisting of a variety of NLP tasks. The proposed evaluation framework encompasses 25 publicly available datasets from diverse enterprise domains like financial services, legal, cyber security, and climate and sustainability. The diverse performance of 13 models across different enterprise tasks highlights the importance of selecting the right model based on the specific requirements of each task. Code and prompts are available on GitHub.
A Dutch Financial Large Language Model
Authors: Sander Noels, Jorne De Blaere, Tijl De Bie
Date: 2024-10-03 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, model

This paper presents FinGEITje, the first Dutch financial Large Language Model (LLM) specifically designed and optimized for various financial tasks. Together with the model, we release a specialized Dutch financial instruction tuning dataset with over 140,000 samples, constructed employing an automated translation and data processing method. The open-source data construction method is provided, facilitating the creation of financial instruction datasets in different languages. To evaluate model performance, the study introduces the first Dutch financial evaluation benchmark, along with an automated evaluation method that utilizes an LLM as an independent evaluator, reducing manual intervention in performance evaluation. The experimental results highlight the superior performance of FinGEITje across five critical Dutch and English financial tasks.
Decoding Emotions: Unveiling Facial Expressions through Acoustic Sensing with Contrastive Attention
Authors: Guangjing Wang, Juexing Wang, Ce Zhou, Weikang Ding, Huacheng Zeng, TianXing Li, Qiben Yan
Date: 2024-10-01 00:00:00
Tasks: Facial Expression Recognition

Expression recognition holds great promise for applications such as content recommendation and mental healthcare by accurately detecting users' emotional states. Traditional methods often rely on cameras or wearable sensors, which raise privacy concerns and add extra device burdens. In addition, existing acoustic-based methods struggle to maintain satisfactory performance when there is a distribution shift between the training dataset and the inference dataset. In this paper, we introduce FacER+, an active acoustic facial expression recognition system, which eliminates the requirement for external microphone arrays. FacER+ extracts facial expression features by analyzing the echoes of near-ultrasound signals emitted between the 3D facial contour and the earpiece speaker on a smartphone. This approach not only reduces background noise but also enables the identification of different expressions from various users with minimal training data. We develop a contrastive external attention-based model to consistently learn expression features across different users, reducing the distribution differences. Extensive experiments involving 20 volunteers, both with and without masks, demonstrate that FacER+ can accurately recognize six common facial expressions with over 90% accuracy in diverse, user-independent real-life scenarios, surpassing the performance of the leading acoustic sensing methods by 10%. FacER+ offers a robust and practical solution for facial expression recognition.
Leveraging generative models to characterize the failure conditions of image classifiers
Authors: Adrien LeCoz, Stéphane Herbin, Faouzi Adjed
Date: 2024-10-01 00:00:00

We address in this work the question of identifying the failure conditions of a given image classifier. To do so, we exploit the capacity of producing controllable distributions of high quality image data made available by recent Generative Adversarial Networks (StyleGAN2): the failure conditions are expressed as directions of strong performance degradation in the generative model latent space. This strategy of analysis is used to discover corner cases that combine multiple sources of corruption, and to compare in more details the behavior of different classifiers. The directions of degradation can also be rendered visually by generating data for better interpretability. Some degradations such as image quality can affect all classes, whereas other ones such as shape are more class-specific. The approach is demonstrated on the MNIST dataset that has been completed by two sources of corruption: noise and blur, and shows a promising way to better understand and control the risks of exploiting Artificial Intelligence components for safety-critical applications.
Rethinking Misalignment in Vision-Language Model Adaptation from a Causal Perspective
Authors: Yanan Zhang, Jiangmeng Li, Lixiang Liu, Wenwen Qiang
Date: 2024-10-01 00:00:00
Tasks: Language Modeling, Language Modelling

Foundational Vision-Language models such as CLIP have exhibited impressive generalization in downstream tasks. However, CLIP suffers from a two-level misalignment issue, i.e., task misalignment and data misalignment, when adapting to specific tasks. Soft prompt tuning has mitigated the task misalignment, yet the data misalignment remains a challenge. To analyze the impacts of the data misalignment, we revisit the pre-training and adaptation processes of CLIP and develop a structural causal model. We discover that while we expect to capture task-relevant information for downstream tasks accurately, the task-irrelevant knowledge impacts the prediction results and hampers the modeling of the true relationships between the images and the predicted classes. As task-irrelevant knowledge is unobservable, we leverage the front-door adjustment and propose Causality-Guided Semantic Decoupling and Classification (CDC) to mitigate the interference of task-irrelevant knowledge. Specifically, we decouple semantics contained in the data of downstream tasks and perform classification based on each semantic. Furthermore, we employ the Dempster-Shafer evidence theory to evaluate the uncertainty of each prediction generated by diverse semantics. Experiments conducted in multiple different settings have consistently demonstrated the effectiveness of CDC.
Interactive Explainable Anomaly Detection for Industrial Settings
Authors: Daniel Gramelt, Timon Höfer, Ute Schmid
Date: 2024-10-01 00:00:00
Tasks: Anomaly Detection

Being able to recognise defects in industrial objects is a key element of quality assurance in production lines. Our research focuses on visual anomaly detection in RGB images. Although Convolutional Neural Networks (CNNs) achieve high accuracies in this task, end users in industrial environments receive the model's decisions without additional explanations. Therefore, it is of interest to enrich the model's outputs with further explanations to increase confidence in the model and speed up anomaly detection. In our work, we focus on (1) CNN-based classification models and (2) the further development of a model-agnostic explanation algorithm for black-box classifiers. Additionally, (3) we demonstrate how we can establish an interactive interface that allows users to further correct the model's output. We present our NearCAIPI Interaction Framework, which improves AI through user interaction, and show how this approach increases the system's trustworthiness. We also illustrate how NearCAIPI can integrate human feedback into an interactive process chain.
AVID: Adapting Video Diffusion Models to World Models
Authors: Marc Rigter, Tarun Gupta, Agrin Hilmkil, Chao Ma
Date: 2024-10-01 00:00:00
Tasks: Decision Making, Sequential Decision Making

Large-scale generative models have achieved remarkable success in a number of domains. However, for sequential decision-making problems, such as robotics, action-labelled data is often scarce and therefore scaling-up foundation models for decision-making remains a challenge. A potential solution lies in leveraging widely-available unlabelled videos to train world models that simulate the consequences of actions. If the world model is accurate, it can be used to optimize decision-making in downstream tasks. Image-to-video diffusion models are already capable of generating highly realistic synthetic videos. However, these models are not action-conditioned, and the most powerful models are closed-source which means they cannot be finetuned. In this work, we propose to adapt pretrained video diffusion models to action-conditioned world models, without access to the parameters of the pretrained model. Our approach, AVID, trains an adapter on a small domain-specific dataset of action-labelled videos. AVID uses a learned mask to modify the intermediate outputs of the pretrained model and generate accurate action-conditioned videos. We evaluate AVID on video game and real-world robotics data, and show that it outperforms existing baselines for diffusion model adaptation.1 Our results demonstrate that if utilized correctly, pretrained video models have the potential to be powerful tools for embodied AI.
GCM-Net: Graph-enhanced Cross-Modal Infusion with a Metaheuristic-Driven Network for Video Sentiment and Emotion Analysis
Authors: Prasad Chaudhari, Aman Kumar, Chandravardhan Singh Raghaw, Mohammad Zia Ur Rehman, Nagendra Kumar
Date: 2024-10-02 00:00:00
Tasks: Emotion Recognition, Graph Sampling, Sentiment Analysis

Sentiment analysis and emotion recognition in videos are challenging tasks, given the diversity and complexity of the information conveyed in different modalities. Developing a highly competent framework that effectively addresses the distinct characteristics across various modalities is a primary concern in this domain. Previous studies on combined multimodal sentiment and emotion analysis often overlooked effective fusion for modality integration, intermodal contextual congruity, optimizing concatenated feature spaces, leading to suboptimal architecture. This paper presents a novel framework that leverages the multi-modal contextual information from utterances and applies metaheuristic algorithms to learn the contributing features for utterance-level sentiment and emotion prediction. Our Graph-enhanced Cross-Modal Infusion with a Metaheuristic-Driven Network (GCM-Net) integrates graph sampling and aggregation to recalibrate the modality features for video sentiment and emotion prediction. GCM-Net includes a cross-modal attention module determining intermodal interactions and utterance relevance. A harmonic optimization module employing a metaheuristic algorithm combines attended features, allowing for handling both single and multi-utterance inputs. To show the effectiveness of our approach, we have conducted extensive evaluations on three prominent multi-modal benchmark datasets, CMU MOSI, CMU MOSEI, and IEMOCAP. The experimental results demonstrate the efficacy of our proposed approach, showcasing accuracies of 91.56% and 86.95% for sentiment analysis on MOSI and MOSEI datasets. We have performed emotion analysis for the IEMOCAP dataset procuring an accuracy of 85.66% which signifies substantial performance enhancements over existing methods.
ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Models
Authors: Mengxue Qu, Xiaodong Chen, Wu Liu, Alicia Li, Yao Zhao
Date: 2024-10-01 00:00:00

Video Temporal Grounding (VTG) aims to ground specific segments within an untrimmed video corresponding to the given natural language query. Existing VTG methods largely depend on supervised learning and extensive annotated data, which is labor-intensive and prone to human biases. To address these challenges, we present ChatVTG, a novel approach that utilizes Video Dialogue Large Language Models (LLMs) for zero-shot video temporal grounding. Our ChatVTG leverages Video Dialogue LLMs to generate multi-granularity segment captions and matches these captions with the given query for coarse temporal grounding, circumventing the need for paired annotation data. Furthermore, to obtain more precise temporal grounding results, we employ moment refinement for fine-grained caption proposals. Extensive experiments on three mainstream VTG datasets, including Charades-STA, ActivityNet-Captions, and TACoS, demonstrate the effectiveness of ChatVTG. Our ChatVTG surpasses the performance of current zero-shot methods.
Deep Adversarial Learning with Activity-Based User Discrimination Task for Human Activity Recognition
Authors: Francisco M. Calatrava-Nicolás, Shoko Miyauchi, Oscar Martinez Mozos
Date: 2024-10-01 00:00:00
Tasks: Activity Recognition, Human Activity Recognition

We present a new adversarial deep learning framework for the problem of human activity recognition (HAR) using inertial sensors worn by people. Our framework incorporates a novel adversarial activity-based discrimination task that addresses inter-person variability-i.e., the fact that different people perform the same activity in different ways. Overall, our proposed framework outperforms previous approaches on three HAR datasets using a leave-one-(person)-out cross-validation (LOOCV) benchmark. Additional results demonstrate that our discrimination task yields better classification results compared to previous tasks within the same adversarial framework.
DyMix: Dynamic Frequency Mixup Scheduler based Unsupervised Domain Adaptation for Enhancing Alzheimer's Disease Identification
Authors: Yooseung Shin, Kwanseok Oh, Heung-Il Suk
Date: 2024-10-02 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Advances in deep learning (DL)-based models for brain image analysis have significantly enhanced the accuracy of Alzheimer's disease (AD) diagnosis, allowing for more timely interventions. Despite these advancements, most current DL models suffer from performance degradation when inferring on unseen domain data owing to the variations in data distributions, a phenomenon known as domain shift. To address this challenge, we propose a novel approach called the dynamic frequency mixup scheduler (DyMix) for unsupervised domain adaptation. Contrary to the conventional mixup technique, which involves simple linear interpolations between predefined data points from the frequency space, our proposed DyMix dynamically adjusts the magnitude of the frequency regions being mixed from the source and target domains. Such an adaptive strategy optimizes the model's capacity to deal with domain variability, thereby enhancing its generalizability across the target domain. In addition, we incorporate additional strategies to further enforce the model's robustness against domain shifts, including leveraging amplitude-phase recombination to ensure resilience to intensity variations and applying self-adversarial learning to derive domain-invariant feature representations. Experimental results on two benchmark datasets quantitatively and qualitatively validated the effectiveness of our DyMix in that we demonstrated its outstanding performance in AD diagnosis compared to state-of-the-art methods.
Segment as You Wish -- Free-Form Language-Based Segmentation for Medical Images
Authors: Longchao Da, Rui Wang, Xiaojian Xu, Parminder Bhatia, Taha Kass-Hout, Hua Wei, Cao Xiao
Date: 2024-10-02 00:00:00
Tasks: Anatomy, Form, Image Segmentation, Medical Image Segmentation, Position, RAG, Segmentation, Semantic Segmentation

Medical imaging is crucial for diagnosing a patient's health condition, and accurate segmentation of these images is essential for isolating regions of interest to ensure precise diagnosis and treatment planning. Existing methods primarily rely on bounding boxes or point-based prompts, while few have explored text-related prompts, despite clinicians often describing their observations and instructions in natural language. To address this gap, we first propose a RAG-based free-form text prompt generator, that leverages the domain corpus to generate diverse and realistic descriptions. Then, we introduce FLanS, a novel medical image segmentation model that handles various free-form text prompts, including professional anatomy-informed queries, anatomy-agnostic position-driven queries, and anatomy-agnostic size-driven queries. Additionally, our model also incorporates a symmetry-aware canonicalization module to ensure consistent, accurate segmentations across varying scan orientations and reduce confusion between the anatomical position of an organ and its appearance in the scan. FLanS is trained on a large-scale dataset of over 100k medical images from 7 public datasets. Comprehensive experiments demonstrate the model's superior language understanding and segmentation precision, along with a deep comprehension of the relationship between them, outperforming SOTA baselines on both in-domain and out-of-domain datasets.
MyData: A Comprehensive Database of Mycetoma Tissue Microscopic Images for Histopathological Analysis
Authors: Hyam Omar Ali, Romain Abraham, Guillaume Desoubeaux, Ahmed Fahal, Clovis Tauber
Date: 2024-10-02 00:00:00

Mycetoma is a chronic and neglected inflammatory disease prevalent in tropical and subtropical regions. It can lead to severe disability and social stigma. The disease is classified into two types based on the causative microorganisms: eumycetoma (fungal) and actinomycetoma (bacterial). Effective treatment strategies depend on accurately identifying the causative agents. Current identification methods include molecular, cytological, and histopathological techniques, as well as grain culturing. Among these, histopathological techniques are considered optimal for use in endemic areas, but they require expert pathologists for accurate identification, which can be challenging in rural areas lacking such expertise. The advent of digital pathology and automated image analysis algorithms offers a potential solution. This report introduces a novel dataset designed for the automated detection and classification of mycetoma using histopathological images. It includes the first database of microscopic images of mycetoma tissue, detailing the entire pipeline from species distribution and patient sampling to acquisition protocols through histological procedures. The dataset consists of images from 142 patients, totalling 864 images, each annotated with binary masks indicating the presence of grains, facilitating both detection and segmentation tasks.
EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
Authors: Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, Kevin Lin, JianFeng Wang, Lijuan Wang, Xin Eric Wang
Date: 2024-10-03 00:00:00
Tasks: 3D scene Editing

Given the steep learning curve of professional 3D software and the time-consuming process of managing large 3D assets, language-guided 3D scene editing has significant potential in fields such as virtual reality, augmented reality, and gaming. However, recent approaches to language-guided 3D scene editing either require manual interventions or focus only on appearance modifications without supporting comprehensive scene layout changes. In response, we propose EditRoom, a unified framework capable of executing a variety of layout edits through natural language commands, without requiring manual intervention. Specifically, EditRoom leverages Large Language Models (LLMs) for command planning and generates target scenes using a diffusion-based method, enabling six types of edits: rotate, translate, scale, replace, add, and remove. To address the lack of data for language-guided 3D scene editing, we have developed an automatic pipeline to augment existing 3D scene synthesis datasets and introduced EditRoom-DB, a large-scale dataset with 83k editing pairs, for training and evaluation. Our experiments demonstrate that our approach consistently outperforms other baselines across all metrics, indicating higher accuracy and coherence in language-guided scene layout editing.