pwc-archive/papers-with-abstracts huggingface dataset:
Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects
Authors: Dominic Lohr, Marc Berges, Abhishek Chugh, Michael Kohlhase, Dennis Müller
Date: 2024-12-05 00:00:00
Tasks: Question Generation, Question-Generation, RAG, Retrieval-augmented Generation

Background: Over the past few decades, the process and methodology of automated question generation (AQG) have undergone significant transformations. Recent progress in generative natural language models has opened up new potential in the generation of educational content. Objectives: This paper explores the potential of large language models (LLMs) for generating computer science questions that are sufficiently annotated for automatic learner model updates, are fully situated in the context of a particular course, and address the cognitive dimension understand. Methods: Unlike previous attempts that might use basic methods like ChatGPT, our approach involves more targeted strategies such as retrieval-augmented generation (RAG) to produce contextually relevant and pedagogically meaningful learning objects. Results and Conclusions: Our results show that generating structural, semantic annotations works well. However, this success was not reflected in the case of relational annotations. The quality of the generated questions often did not meet educational standards, highlighting that although LLMs can contribute to the pool of learning materials, their current level of performance requires significant human intervention to refine and validate the generated content.
Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation
Authors: Xuying Li, Zhuo Li, Yuji Kosuga, Yasuhiro Yoshida, Victor Bian
Date: 2024-12-05 00:00:00
Tasks: Fairness, RAG

AI agents, powered by large language models (LLMs), have transformed human-computer interactions by enabling seamless, natural, and context-aware communication. While these advancements offer immense utility, they also inherit and amplify inherent safety risks such as bias, fairness, hallucinations, privacy breaches, and a lack of transparency. This paper investigates a critical vulnerability: adversarial attacks targeting the LLM core within AI agents. Specifically, we test the hypothesis that a deceptively simple adversarial prefix, such as \textit{Ignore the document}, can compel LLMs to produce dangerous or unintended outputs by bypassing their contextual safeguards. Through experimentation, we demonstrate a high attack success rate (ASR), revealing the fragility of existing LLM defenses. These findings emphasize the urgent need for robust, multi-layered security measures tailored to mitigate vulnerabilities at the LLM level and within broader agent-based architectures.
Back-filling Missing Data When Predicting Domestic Electricity Consumption From Smart Meter Data
Authors: Xianjuan Chen, Shuxiang Cai, Alan F. Smeaton
Date: 2024-11-17 00:00:00

This study uses data from domestic electricity smart meters to estimate annual electricity bills for a whole year. We develop a method for back-filling data smart meter for up to six missing months for users who have less than one year of smart meter data, ensuring reliable estimates of annual consumption. We identify five distinct electricity consumption user profiles for homes based on day, night, and peak usage patterns, highlighting the economic advantages of Time-of-Use (ToU) tariffs over fixed tariffs for most users, especially those with higher nighttime consumption. Ultimately, the results of this study empowers consumers to manage their energy use effectively and to make informed choices regarding electricity tariff plans.
Ethical Challenges and Evolving Strategies in the Integration of Artificial Intelligence into Clinical Practice
Authors: Ellison B. Weiner, Irene Dankwa-Mullan, William A. Nelson, Saeed Hassanpour
Date: 2024-11-18 00:00:00
Tasks: Fairness

Artificial intelligence (AI) has rapidly transformed various sectors, including healthcare, where it holds the potential to revolutionize clinical practice and improve patient outcomes. However, its integration into medical settings brings significant ethical challenges that need careful consideration. This paper examines the current state of AI in healthcare, focusing on five critical ethical concerns: justice and fairness, transparency, patient consent and confidentiality, accountability, and patient-centered and equitable care. These concerns are particularly pressing as AI systems can perpetuate or even exacerbate existing biases, often resulting from non-representative datasets and opaque model development processes. The paper explores how bias, lack of transparency, and challenges in maintaining patient trust can undermine the effectiveness and fairness of AI applications in healthcare. In addition, we review existing frameworks for the regulation and deployment of AI, identifying gaps that limit the widespread adoption of these systems in a just and equitable manner. Our analysis provides recommendations to address these ethical challenges, emphasizing the need for fairness in algorithm design, transparency in model decision-making, and patient-centered approaches to consent and data privacy. By highlighting the importance of continuous ethical scrutiny and collaboration between AI developers, clinicians, and ethicists, we outline pathways for achieving more responsible and inclusive AI implementation in healthcare. These strategies, if adopted, could enhance both the clinical value of AI and the trustworthiness of AI systems among patients and healthcare professionals, ensuring that these technologies serve all populations equitably.
Towards a Practical Ethics of Generative AI in Creative Production Processes
Authors: Geert Hofman
Date: 2024-11-18 00:00:00
Tasks: Ethics, Navigate

The increasing integration of artificial intelligence into various domains, including design and creative processes, raises significant ethical questions. While AI ethics is often examined from the perspective of technology developers, less attention has been paid to the practical ethical considerations faced by technology users, particularly in design contexts. This paper introduces a framework for addressing ethical challenges in creative production processes, such as the Double Diamond design model. Drawing on six major ethical theories - virtue ethics, deontology, utilitarianism, contract theory, care ethics, and existentialism - we develop a "compass" to navigate and reflect on the ethical dimensions of AI in design. The framework highlights the importance of responsibility, anticipation, and reflection across both the AI lifecycle and each stage of the creative process. We argue that by adopting a playful and exploratory approach to AI, while remaining anchored in core ethical principles, designers can responsibly harness the potential of AI technologies without overburdening or compromising their creative processes.
ParetoFlow: Guided Flows in Multi-Objective Optimization
Authors: Ye Yuan, Can Chen, Christopher Pal, Xue Liu
Date: 2024-12-04 00:00:00
Tasks: Bayesian Optimization, Evolutionary Algorithms

In offline multi-objective optimization (MOO), we leverage an offline dataset of designs and their associated labels to simultaneously minimize multiple objectives. This setting more closely mirrors complex real-world problems compared to single-objective optimization. Recent works mainly employ evolutionary algorithms and Bayesian optimization, with limited attention given to the generative modeling capabilities inherent in such data. In this study, we explore generative modeling in offline MOO through flow matching, noted for its effectiveness and efficiency. We introduce ParetoFlow, specifically designed to guide flow sampling to approximate the Pareto front. Traditional predictor (classifier) guidance is inadequate for this purpose because it models only a single objective. In response, we propose a multi-objective predictor guidance module that assigns each sample a weight vector, representing a weighted distribution across multiple objective predictions. A local filtering scheme is introduced to address non-convex Pareto fronts. These weights uniformly cover the entire objective space, effectively directing sample generation towards the Pareto front. Since distributions with similar weights tend to generate similar samples, we introduce a neighboring evolution module to foster knowledge sharing among neighboring distributions. This module generates offspring from these distributions, and selects the most promising one for the next iteration. Our method achieves state-of-the-art performance across various tasks.
PDG2Seq: Periodic Dynamic Graph to Sequence Model for Traffic Flow Prediction
Authors: Jin Fan, Weng, Wenchao, Qikai Chen, Huifeng Wu, Jia Wu
Date: 2024-12-05 00:00:00
Proceeding: Neural Networks 2024 12
Tasks: feature selection, Graph-to-Sequence, Traffic Prediction

Traffic flow prediction is the foundation of intelligent traffic management systems. Current methods prioritize the development of intricate models to capture spatio-temporal correlations, yet they often neglect the exploitation of latent features within traffic flow. Firstly, the correlation among different road nodes exhibits dynamism rather than remaining static. Secondly, traffic data exhibits evident periodicity, yet current research lacks the exploration and utilization of periodic features. Lastly, current models typically rely solely on historical data for modeling, resulting in the limitation of accurately capturing future trend changes in traffic flow. To address these findings, this paper proposes a Periodic Dynamic Graph to Sequence Model (PDG2Seq) for traffic flow prediction. PDG2Seq consists of the Periodic Feature Selection Module (PFSM) and the Periodic Dynamic Graph Convolutional Gated Recurrent Unit (PDCGRU) to further extract the spatio-temporal features of the dynamic real-time traffic. The PFSM extracts learned periodic features using time points as indices, while the PDCGRU leverages the extracted periodic features from the PFSM and dynamic features from traffic flow to generate a Periodic Dynamic Graph for extracting spatio-temporal features. In the decoding phase, PDG2Seq utilizes periodic features corresponding to the prediction target to capture future trend changes, leading to more accurate predictions. Comprehensive experiments conducted on four large-scale datasets substantiate the superiority of PDG2Seq over existing state-of-the-art baselines. Related codes are available at https://github.com/wengwenchao123/PDG2Seq.
Spectral Energy Distributions of Hard X-ray selected AGNs in the XMDS Survey
Authors: M. Polletta, M. Tajer, L. Maraschi, G. Trinchieri, C. J. Lonsdale, L. Chiappetti, S. Andreon, M. Pierre, O. Le Fevre, G. Zamorani, D. Maccagni, O. Garcet, J. Surdej, A. Franceschini, D. Alloin, D. L. Shupe, J. A. Surace, F. Fang, M. Rowan-Robinson, H. E. Smith, L. Tresse
Date: 2007-03-12 00:00:00

We present the spectral energy distributions (SEDs) of a hard X-ray selected sample. The sample contains 136 sources with F(2-10 keV)>10^-14 erg/cm^2/s and 132 are AGNs. The sources are detected in a 1 square degree area of the XMM-Newton-Medium Deep Survey where optical data from the VVDS, CFHTLS surveys, and infrared data from the SWIRE survey are available. Based on a SED fitting technique we derive photometric redshifts with sigma(1+z)=0.11 and 6% of outliers and identify AGN signatures in 83% of the objects. This fraction is higher than derived when a spectroscopic classification is available. The remaining 17+9-6% of AGNs shows star-forming galaxy SEDs (SF class). The sources with AGN signatures are divided in two classes, AGN1 (33+6-1%) and AGN2 (50+6-11). The AGN1 and AGN2 classes include sources whose SEDs are fitted by type 1 and type 2 AGN templates, respectively. On average, AGN1s show soft X-ray spectra, consistent with being unabsorbed, while AGN2s and SFs show hard X-ray spectra, consistent with being absorbed. The analysis of the average SEDs as a function of X-ray luminosity shows a reddening of the IR SEDs, consistent with a decreasing contribution from the host galaxy at higher luminosities. The AGNs in the SF classes are likely obscured in the mid-infrared, as suggested by their low L(3-20micron)/Lcorr(0.5-10 keV) ratios. We confirm the previously found correlation for AGNs between the radio luminosity and the X-ray and the mid-infrared luminosities. The X-ray-radio correlation can be used to identify heavily absorbed AGNs. However, the estimated radio fluxes for the missing AGN population responsible for the bulk of the background at E>10 keV are too faint to be detected even in the deepest current radio surveys.
Learning to Detect Attended Objects in Cultural Sites with Gaze Signals and Weak Object Supervision
Authors: Michele Mazzamuto, Francesco Ragusa, Antonino Furnari, Giovanni Maria Farinella
Date: 2024-04-23 00:00:00
Proceeding: Journal on Computing and Cultural Heritage 2024 4
Tasks: Object, object-detection, Object Detection

Cultural sites such as museums and monuments are popular tourist destinations worldwide. Visitors come to these places to learn about the cultures, histories and arts of a particular region or country. However, for many cultural sites, traditional visiting approaches are limited and may fail to engage visitors. To enhance visitors' experiences, previous works have explored how wearable devices can be exploited in this context. Among the many functions that these devices can offer, understanding which artwork or detail the user is attending to is fundamental to provide additional information on the observed artworks, understand the visitor's tastes and provide recommendations. This motivates the development of algorithms for understanding visitor attention from egocentric images. We considered the attended object detection task, which involves detecting and recognizing the object observed by the camera wearer, from an input RGB image and gaze signals. To study the problem, we collect a dataset of egocentric images collected by subjects visiting a museum. Since collecting and labeling data in cultural sites for real applications is a time-consuming problem, we present a study comparing unsupervised, weakly supervised, and fully supervised approaches for attended object detection. We evaluate the considered approaches on the collected dataset, assessing also the impact of training models on external datasets such as COCO and EGO-CH. The experiments show that weakly supervised approaches requiring only a 2D point label related to the gaze can be an effective alternative to fully supervised approaches for attended object detection.
Hard-Label Black-Box Attacks on 3D Point Clouds
Authors: Daizong Liu, Yunbo Tao, Pan Zhou, Wei Hu
Date: 2024-11-30 00:00:00

With the maturity of depth sensors in various 3D safety-critical applications, 3D point cloud models have been shown to be vulnerable to adversarial attacks. Almost all existing 3D attackers simply follow the white-box or black-box setting to iteratively update coordinate perturbations based on back-propagated or estimated gradients. However, these methods are hard to deploy in real-world scenarios (no model details are provided) as they severely rely on parameters or output logits of victim models. To this end, we propose point cloud attacks from a more practical setting, i.e., hard-label black-box attack, in which attackers can only access the prediction label of 3D input. We introduce a novel 3D attack method based on a new spectrum-aware decision boundary algorithm to generate high-quality adversarial samples. In particular, we first construct a class-aware model decision boundary, by developing a learnable spectrum-fusion strategy to adaptively fuse point clouds of different classes in the spectral domain, aiming to craft their intermediate samples without distorting the original geometry. Then, we devise an iterative coordinate-spectrum optimization method with curvature-aware boundary search to move the intermediate sample along the decision boundary for generating adversarial point clouds with trivial perturbations. Experiments demonstrate that our attack competitively outperforms existing white/black-box attackers in terms of attack performance and adversary quality.
SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments
Authors: Yue Cao, Yun Xing, Jie Zhang, Di Lin, Tianwei Zhang, Ivor Tsang, Yang Liu, Qing Guo
Date: 2024-11-28 00:00:00
Proceeding: CVPR 2025 1
Tasks: Adversarial Text, Scene Understanding

Large vision-language models (LVLMs) have shown remarkable capabilities in interpreting visual content. While existing works demonstrate these models' vulnerability to deliberately placed adversarial texts, such texts are often easily identifiable as anomalous. In this paper, we present the first approach to generate scene-coherent typographic adversarial attacks that mislead advanced LVLMs while maintaining visual naturalness through the capability of the LLM-based agent. Our approach addresses three critical questions: what adversarial text to generate, where to place it within the scene, and how to integrate it seamlessly. We propose a training-free, multi-modal LLM-driven scene-coherent typographic adversarial planning (SceneTAP) that employs a three-stage process: scene understanding, adversarial planning, and seamless integration. The SceneTAP utilizes chain-of-thought reasoning to comprehend the scene, formulate effective adversarial text, strategically plan its placement, and provide detailed instructions for natural integration within the image. This is followed by a scene-coherent TextDiffuser that executes the attack using a local diffusion mechanism. We extend our method to real-world scenarios by printing and placing generated patches in physical environments, demonstrating its practical implications. Extensive experiments show that our scene-coherent adversarial text successfully misleads state-of-the-art LVLMs, including ChatGPT-4o, even after capturing new images of physical setups. Our evaluations demonstrate a significant increase in attack success rates while maintaining visual naturalness and contextual appropriateness. This work highlights vulnerabilities in current vision-language models to sophisticated, scene-coherent adversarial attacks and provides insights into potential defense mechanisms.
Dynamic Token Selection for Aerial-Ground Person Re-Identification
Authors: Yuhai Wang, Maryam Pishgar
Date: 2024-11-30 00:00:00
Tasks: Computational Efficiency, Person Re-Identification, Retrieval

Aerial-Ground Person Re-identification (AGPReID) holds significant practical value but faces unique challenges due to pronounced variations in viewing angles, lighting conditions, and background interference. Traditional methods, often involving a global analysis of the entire image, frequently lead to inefficiencies and susceptibility to irrelevant data. In this paper, we propose a novel Dynamic Token Selective Transformer (DTST) tailored for AGPReID, which dynamically selects pivotal tokens to concentrate on pertinent regions. Specifically, we segment the input image into multiple tokens, with each token representing a unique region or feature within the image. Using a Top-k strategy, we extract the k most significant tokens that contain vital information essential for identity recognition. Subsequently, an attention mechanism is employed to discern interrelations among diverse tokens, thereby enhancing the representation of identity features. Extensive experiments on benchmark datasets showcases the superiority of our method over existing works. Notably, on the CARGO dataset, our proposed method gains 1.18% mAP improvements when compared to the second place. In addition, we comprehensively analyze the impact of different numbers of tokens, token insertion positions, and numbers of heads on model performance.
Explainable Artificial Intelligence for Medical Applications: A Review
Authors: Qiyang Sun, Alican Akman, Björn W. Schuller
Date: 2024-11-15 00:00:00
Tasks: Computed Tomography (CT), Decision Making, Disease Prediction, Explainable artificial intelligence, Explainable Artificial Intelligence (XAI), Management

The continuous development of artificial intelligence (AI) theory has propelled this field to unprecedented heights, owing to the relentless efforts of scholars and researchers. In the medical realm, AI takes a pivotal role, leveraging robust machine learning (ML) algorithms. AI technology in medical imaging aids physicians in X-ray, computed tomography (CT) scans, and magnetic resonance imaging (MRI) diagnoses, conducts pattern recognition and disease prediction based on acoustic data, delivers prognoses on disease types and developmental trends for patients, and employs intelligent health management wearable devices with human-computer interaction technology to name but a few. While these well-established applications have significantly assisted in medical field diagnoses, clinical decision-making, and management, collaboration between the medical and AI sectors faces an urgent challenge: How to substantiate the reliability of decision-making? The underlying issue stems from the conflict between the demand for accountability and result transparency in medical scenarios and the black-box model traits of AI. This article reviews recent research grounded in explainable artificial intelligence (XAI), with an emphasis on medical practices within the visual, audio, and multimodal perspectives. We endeavour to categorise and synthesise these practices, aiming to provide support and guidance for future researchers and healthcare professionals.
World-Map Misalignment Detection for Visual Navigation Systems
Authors: Rosario Forte, Michele Mazzamuto, Francesco Ragusa, Giovanni Maria Farinella, Antonino Furnari
Date: 2024-01-01 00:00:00
Proceeding: 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2024 1
Tasks: Visual Navigation

We consider the problem of inferring when the internal map of an indoor navigation system is misaligned with respect to the real world (world-map misalignment), which can lead to misleading directions given to the user. We note that world-map misalignment can be predicted from an RGB image of the environment and the floor segmentation mask obtained from the internal map of the navigation system. Since collecting and labelling large amounts of real data is expensive, we developed a tool to simulate human navigation, which is used to generate automatically labelled synthetic data from 3D models of environments. Thanks to this tool, we generate a dataset considering 15 different environments, which is complemented by a small set of videos acquired in a real-world scenario and manually labelled for validation purposes. We hence benchmark an approach based on different ResNet18 configurations and compare their results on both synthetic and real images. We achieved an F1 score of 92.37% in the synthetic domain and 75.42% on the proposed real dataset using our best approach. While the results are promising, we also note that the proposed problem is challenging, due to the domain shift between synthetic and real data, and the difficulty in acquiring real data. The dataset and the developed tool are publicly available to encourage research on the topic at the following URL: https://github.com/fpv-iplab/WMM-detection-for-visual-navigation-systems