pwc-archive/papers-with-abstracts huggingface dataset:
$\textit{Revelio}$: Interpreting and leveraging semantic information in diffusion models
Authors: Dahye Kim, Xavier Thomas, Deepti Ghadiyaram
Date: 2024-11-23 00:00:00
Tasks: Denoising, Language Modeling, Language Modelling, Representation Learning, Transfer Learning

We study $\textit{how}$ rich visual semantic information is represented within various layers and denoising timesteps of different diffusion architectures. We uncover monosemantic interpretable features by leveraging k-sparse autoencoders (k-SAE). We substantiate our mechanistic interpretations via transfer learning using light-weight classifiers on off-the-shelf diffusion models' features. On $4$ datasets, we demonstrate the effectiveness of diffusion features for representation learning. We provide in-depth analysis of how different diffusion architectures, pre-training datasets, and language model conditioning impacts visual representation granularity, inductive biases, and transfer learning capabilities. Our work is a critical step towards deepening interpretability of black-box diffusion models. Code and visualizations available at: https://github.com/revelio-diffusion/revelio
Multi-task learning-based temporal pattern matching network for guitar tablature transcription
Authors: Taehyeon Kim, Man-Je Kim, Chang Wook Ahn
Date: 2025-04-03 00:00:00
Proceeding: Neural Computing and Applications 2025 4
Tasks: Multi-Task Learning, Music Transcription

Guitar tablature transcription poses unique challenges in automatic music transcription, as it requires capturing both pitch and string usage on a multi-string instrument with various expressive techniques. While guitar tablature is widely used by guitarists in the music field, neural architecture modeling for this task remains underexplored, particularly in accurately mapping pitches to their respective strings. In this work, we propose a multi-task learning-based temporal pattern-matching network (TPMNet) that effectively captures temporal information from guitar recordings, improving the alignment of predicted results. The key contribution of this work is the advancement of neural network architecture, leading to notable improvements in prediction performance for guitar tablature transcription. Additionally, we explored the optimal pooling layer selection method tailored to different tasks, addressing a long-confusing problem in the field. TPMNet’s efficacy was validated through experiments on the GuitarSet dataset, and its generalizability was confirmed via cross-evaluation with the EGDB dataset.
Interpretable Automatic Rosacea Detection with Whitened Cosine Similarity
Authors: Chengyu Yang, Chengjun Liu
Date: 2025-04-10 00:00:00

According to the National Rosacea Society, approximately sixteen million Americans suffer from rosacea, a common skin condition that causes flushing or long-term redness on a person's face. To increase rosacea awareness and to better assist physicians to make diagnosis on this disease, we propose an interpretable automatic rosacea detection method based on whitened cosine similarity in this paper. The contributions of the proposed methods are three-fold. First, the proposed method can automatically distinguish patients suffering from rosacea from people who are clean of this disease with a significantly higher accuracy than other methods in unseen test data, including both classical deep learning and statistical methods. Second, the proposed method addresses the interpretability issue by measuring the similarity between the test sample and the means of two classes, namely the rosacea class versus the normal class, which allows both medical professionals and patients to understand and trust the results. And finally, the proposed methods will not only help increase awareness of rosacea in the general population, but will also help remind patients who suffer from this disease of possible early treatment, as rosacea is more treatable in its early stages. The code and data are available at https://github.com/chengyuyang-njit/ICCRD-2025. The code and data are available at https://github.com/chengyuyang-njit/ICCRD-2025.
RAG-VR: Leveraging Retrieval-Augmented Generation for 3D Question Answering in VR Environments
Authors: Shiyi Ding, Ying Chen
Date: 2025-04-11 00:00:00
Tasks: Answer Generation, Question Answering, RAG, Retrieval, Retrieval-augmented Generation

Recent advances in large language models (LLMs) provide new opportunities for context understanding in virtual reality (VR). However, VR contexts are often highly localized and personalized, limiting the effectiveness of general-purpose LLMs. To address this challenge, we present RAG-VR, the first 3D question-answering system for VR that incorporates retrieval-augmented generation (RAG), which augments an LLM with external knowledge retrieved from a localized knowledge database to improve the answer quality. RAG-VR includes a pipeline for extracting comprehensive knowledge about virtual environments and user conditions for accurate answer generation. To ensure efficient retrieval, RAG-VR offloads the retrieval process to a nearby edge server and uses only essential information during retrieval. Moreover, we train the retriever to effectively distinguish among relevant, irrelevant, and hard-to-differentiate information in relation to questions. RAG-VR improves answer accuracy by 17.9%-41.8% and reduces end-to-end latency by 34.5%-47.3% compared with two baseline systems.
Light-YOLOv8-Flame: A Lightweight High-Performance Flame Detection Algorithm
Authors: Jiawei Lan, Zhibiao Wang, Haoyang Yu, Ye Tao, Wenhua Cui
Date: 2025-04-11 00:00:00
Tasks: Fire Detection

Fire detection algorithms, particularly those based on computer vision, encounter significant challenges such as high computational costs and delayed response times, which hinder their application in real-time systems. To address these limitations, this paper introduces Light-YOLOv8-Flame, a lightweight flame detection algorithm specifically designed for fast and efficient real-time deployment. The proposed model enhances the YOLOv8 architecture through the substitution of the original C2f module with the FasterNet Block module. This new block combines Partial Convolution (PConv) and Convolution (Conv) layers, reducing both computational complexity and model size. A dataset comprising 7,431 images, representing both flame and non-flame scenarios, was collected and augmented for training purposes. Experimental findings indicate that the modified YOLOv8 model achieves a 0.78% gain in mean average precision (mAP) and a 2.05% boost in recall, while reducing the parameter count by 25.34%, with only a marginal decrease in precision by 0.82%. These findings highlight that Light-YOLOv8-Flame offers enhanced detection performance and speed, making it well-suited for real-time fire detection on resource-constrained devices.
A Time-Varying Information Measure for Tracking Dynamics of Neural Codes in a Neural Ensemble
Authors: Mohammad R. Rezaei, Milos R. Popovic, and Milad Lankarany
Date: 2020-08-22 00:00:00
Proceeding: Entropy 2020 8
Tasks: Decoder

The amount of information that differentially correlated spikes in a neural ensemble carry is not the same; the information of different types of spikes is associated with different features of the stimulus. By calculating a neural ensemble’s information in response to a mixed stimulus comprising slow and fast signals, we show that the entropy of synchronous and asynchronous spikes are different, and their probability distributions are distinctively separable. We further show that these spikes carry a different amount of information. We propose a time-varying entropy (TVE) measure to track the dynamics of a neural code in an ensemble of neurons at each time bin. By applying the TVE to a multiplexed code, we show that synchronous and asynchronous spikes carry information in different time scales. Finally, a decoder based on the Kalman filtering approach is developed to reconstruct the stimulus from the spikes. We demonstrate that slow and fast features of the stimulus can be entirely reconstructed when this decoder is applied to asynchronous and synchronous spikes, respectively. The significance of this work is that the TVE can identify different types of information (for example, corresponding to synchronous and asynchronous spikes) that might simultaneously exist in a neural code.
Task Memory Engine (TME): A Structured Memory Framework with Graph-Aware Extensions for Multi-Step LLM Agent Tasks
Authors: Ye Ye
Date: 2025-04-11 00:00:00

Large Language Models (LLMs) are increasingly used as autonomous agents for multi-step tasks. However, most existing frameworks fail to maintain a structured understanding of the task state, often relying on linear prompt concatenation or shallow memory buffers. This leads to brittle performance, frequent hallucinations, and poor long-range coherence. In this work, we propose the Task Memory Engine (TME), a lightweight and structured memory module that tracks task execution using a hierarchical Task Memory Tree (TMT). Each node in the tree corresponds to a task step, storing relevant input, output, status, and sub-task relationships. We introduce a prompt synthesis method that dynamically generates LLM prompts based on the active node path, significantly improving execution consistency and contextual grounding. Through case studies and comparative experiments on multi-step agent tasks, we demonstrate that TME leads to better task completion accuracy and more interpretable behavior with minimal implementation overhead. A reference implementation of the core TME components is available at https://github.com/biubiutomato/TME-Agent, including basic examples and structured memory integration. While the current implementation uses a tree-based structure, TME is designed to be graph-aware, supporting reusable substeps, converging task paths, and shared dependencies. This lays the groundwork for future DAG-based memory architectures.
Towards Efficient and Robust Moment Retrieval System: A Unified Framework for Multi-Granularity Models and Temporal Reranking
Authors: Huu-Loc Tran, Tinh-Anh Nguyen-Nhu, Huu-Phong Phan-Nguyen, Tien-Huy Nguyen, Nhat-Minh Nguyen-Dich, Anh Dao, Huy-Duc Do, Quan Nguyen, Hoang M. Le, Quang-Vinh Dinh
Date: 2025-04-11 00:00:00
Tasks: Moment Retrieval, Question Answering, Reranking, Retrieval, Video Retrieval, Video Understanding

Long-form video understanding presents significant challenges for interactive retrieval systems, as conventional methods struggle to process extensive video content efficiently. Existing approaches often rely on single models, inefficient storage, unstable temporal search, and context-agnostic reranking, limiting their effectiveness. This paper presents a novel framework to enhance interactive video retrieval through four key innovations: (1) an ensemble search strategy that integrates coarse-grained (CLIP) and fine-grained (BEIT3) models to improve retrieval accuracy, (2) a storage optimization technique that reduces redundancy by selecting representative keyframes via TransNetV2 and deduplication, (3) a temporal search mechanism that localizes video segments using dual queries for start and end points, and (4) a temporal reranking approach that leverages neighboring frame context to stabilize rankings. Evaluated on known-item search and question-answering tasks, our framework demonstrates substantial improvements in retrieval precision, efficiency, and user interpretability, offering a robust solution for real-world interactive video retrieval applications.
TorchFX: A modern approach to Audio DSP with PyTorch and GPU acceleration
Authors: Matteo Spanio, Antonio Rodà
Date: 2025-04-11 00:00:00
Tasks: Audio Signal Processing, Benchmarking, GPU

The burgeoning complexity and real-time processing demands of audio signals necessitate optimized algorithms that harness the computational prowess of Graphics Processing Units (GPUs). Existing Digital Signal Processing (DSP) libraries often fall short in delivering the requisite efficiency and flexibility, particularly in integrating Artificial Intelligence (AI) models. In response, we introduce TorchFX: a GPU-accelerated Python library for DSP, specifically engineered to facilitate sophisticated audio signal processing. Built atop the PyTorch framework, TorchFX offers an Object-Oriented interface that emulates the usability of torchaudio, enhancing functionality with a novel pipe operator for intuitive filter chaining. This library provides a comprehensive suite of Finite Impulse Response (FIR) and Infinite Impulse Response (IIR) filters, with a focus on multichannel audio files, thus facilitating the integration of DSP and AI-based approaches. Our benchmarking results demonstrate significant efficiency gains over traditional libraries like SciPy, particularly in multichannel contexts. Despite current limitations in GPU compatibility, ongoing developments promise broader support and real-time processing capabilities. TorchFX aims to become a useful tool for the community, contributing to innovation and progress in DSP with GPU acceleration. TorchFX is publicly available on GitHub at https://github.com/matteospanio/torchfx.
EquiNO: A Physics-Informed Neural Operator for Multiscale Simulations
Authors: Hamidreza Eivazi, Jendrik-Alexander Tröger, Stefan Wittek, Stefan Hartmann, Andreas Rausch
Date: 2025-03-27 00:00:00
Tasks: Operator learning, Uncertainty Quantification

Multiscale problems are ubiquitous in physics. Numerical simulations of such problems by solving partial differential equations (PDEs) at high resolution are computationally too expensive for many-query scenarios, e.g., uncertainty quantification, remeshing applications, topology optimization, and so forth. This limitation has motivated the application of data-driven surrogate models, where the microscale computations are $\textit{substituted}$ with a surrogate, usually acting as a black-box mapping between macroscale quantities. These models offer significant speedups but struggle with incorporating microscale physical constraints, such as the balance of linear momentum and constitutive models. In this contribution, we propose Equilibrium Neural Operator (EquiNO) as a $\textit{complementary}$ physics-informed PDE surrogate for predicting microscale physics and compare it with variational physics-informed neural and operator networks. Our framework, applicable to the so-called multiscale FE$^{\,2}\,$ computations, introduces the FE-OL approach by integrating the finite element (FE) method with operator learning (OL). We apply the proposed FE-OL approach to quasi-static problems of solid mechanics. The results demonstrate that FE-OL can yield accurate solutions even when confronted with a restricted dataset during model development. Our results show that EquiNO achieves speedup factors exceeding 8000-fold compared to traditional methods and offers an optimal balance between data-driven and physics-based strategies.
SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories
Authors: Zhu, Muzhi; Tian, Yuzhuo; Chen, Hao; Zhou, Chunluan; Guo, Qingpei; Liu, Yang; Yang, Ming; Shen, Chunhua
Date: 2025-03-11 00:00:00
Tasks: Decision Making, Interactive Segmentation, Referring Expression Segmentation, Segmentation, Visual Grounding, Visual Question Answering (VQA), Visual Reasoning

While MLLMs have demonstrated adequate image understanding capabilities, they still struggle with pixel-level comprehension, limiting their practical applications. Current evaluation tasks like VQA and visual grounding remain too coarse to assess fine-grained pixel comprehension accurately. Though segmentation is foundational for pixel-level understanding, existing methods often require MLLMs to generate implicit tokens, decoded through external pixel decoders. This approach disrupts the MLLM's text output space, potentially compromising language capabilities and reducing flexibility and extensibility, while failing to reflect the model's intrinsic pixel-level understanding.
Thus, we introduce the Human-Like Mask Annotation Task (HLMAT), a new paradigm where MLLMs mimic human annotators using interactive segmentation tools. Modeling segmentation as a multi-step Markov Decision Process, HLMAT enables MLLMs to iteratively generate text-based click points, achieving high-quality masks without architectural changes or implicit tokens. Through this setup, we develop SegAgent, a model fine-tuned on human-like annotation trajectories, which achieves performance comparable to state-of-the-art (SOTA) methods and supports additional tasks like mask refinement and annotation filtering.
HLMAT provides a protocol for assessing fine-grained pixel understanding in MLLMs and introduces a vision-centric, multi-step decision-making task that facilitates exploration of MLLMs' visual reasoning abilities. Our adaptations of policy improvement method StaR and PRM-guided tree search further enhance model robustness in complex segmentation tasks, laying a foundation for future advancements in fine-grained visual perception and multi-step decision-making for MLLMs.
mixEEG: Enhancing EEG Federated Learning for Cross-subject EEG Classification with Tailored mixup
Authors: Xuan-Hao Liu, Bao-liang Lu, Wei-Long Zheng
Date: 2025-04-07 00:00:00
Tasks: Domain Adaptation, EEG, Emotion Recognition, Federated Learning, Pseudo Label

The cross-subject electroencephalography (EEG) classification exhibits great challenges due to the diversity of cognitive processes and physiological structures between different subjects. Modern EEG models are based on neural networks, demanding a large amount of data to achieve high performance and generalizability. However, privacy concerns associated with EEG pose significant limitations to data sharing between different hospitals and institutions, resulting in the lack of large dataset for most EEG tasks. Federated learning (FL) enables multiple decentralized clients to collaboratively train a global model without direct communication of raw data, thus preserving privacy. For the first time, we investigate the cross-subject EEG classification in the FL setting. In this paper, we propose a simple yet effective framework termed mixEEG. Specifically, we tailor the vanilla mixup considering the unique properties of the EEG modality. mixEEG shares the unlabeled averaged data of the unseen subject rather than simply sharing raw data under the domain adaptation setting, thus better preserving privacy and offering an averaged label as pseudo-label. Extensive experiments are conducted on an epilepsy detection and an emotion recognition dataset. The experimental result demonstrates that our mixEEG enhances the transferability of global model for cross-subject EEG classification consistently across different datasets and model architectures. Code is published at: https://github.com/XuanhaoLiu/mixEEG.
Sphingolipids control dermal fibroblast heterogeneity
Authors: Laura Capolupo, Irina Khven, Alex R. Lederer, Luigi Mazzeo, Galina Glousker, Sylvia Ho, Francesco Russo, Jonathan Paz Montoya, Dhaka R. Bhandari, Andrew P. Bowman, Shane R. Ellis, Romain Guiet, Olivier Burri, Johanna Detzner, Johannes Muthing, Krisztian Homicsko, François Kuonen, Michel Gilliet, Bernhard Spengler, Ron M. A. Heeren, G. Paolo Dotto, Gioele La Manno*, Giovanni D’Angelo
Date: 2022-04-15 00:00:00
Proceeding: Science 2022 4

External signals (e.g., hormones, cytokines, and growth factors) and cell-autonomous properties (e.g., the transcriptional and metabolic states of individual cells) concur to determine cell-fate decisions. Although the mode of action of external signals has been detailed extensively in decades of intense research, the molecular bases of cell-autonomous contribution to cell-fate decisions have been traditionally more elusive. Lipids are fundamental constituents of all living beings. They participate in energy metabolism, account for the assembly of biological membranes, act as signaling molecules, and interact with proteins to influence their function and intracellular distribution. Eukaryotic cells produce thousands of different lipids, each endowed with peculiar structural features and contributing to specific biological functions. With the development of lipidomics, we can now understand the lipid compositional complexity of cells and start making sense of lipidome dynamics. Lipidomes indeed vary among cell types and are reprogrammed in differentiation events. However, whether and how lipidome remodeling assists changes in cell identity is not understood.
Fast Reconfiguration of LC-RISs: Modeling and Algorithm Design
Authors: Mohamadreza Delbari, Robin Neuder, Alejandro Jiménez-Sáez, Arash Asadi, Vahid Jamali
Date: 2025-04-11 00:00:00

LC technology is a promising hardware solution for realizing extremely large RISs due to its advantages in cost-effectiveness, scalability, energy efficiency, and continuous phase shift tunability. However, the slow response time of the LC cells, especially in comparison to the silicon-based alternatives like radio frequency switches and PIN diodes, limits the performance. This limitation becomes particularly relevant in TDMA applications where RIS must sequentially serve users in different locations, as the phase-shifting response time of LC cells can constrain system performance. This paper addresses the slow phase-shifting limitation of LC by developing a physics-based model for the time response of an LC unit cell and proposing a novel phase-shift design framework to reduce the transition time. Specifically, exploiting the fact that LC-RIS at milimeter wave bands have a large electric aperture, we optimize the LC phase shifts based on user locations, eliminating the need for full channel state information and minimizing reconfiguration overhead. Moreover, instead of focusing on a single point, the RIS phase shifters are designed to optimize coverage over an area. This enhances communication reliability for mobile users and mitigates performance degradation due to user location estimation errors. The proposed design minimizes the transition time between configurations, a critical requirement for TDMA schemes. Our analysis reveals that the impact of RIS reconfiguration time on system throughput becomes particularly significant when TDMA intervals are comparable to the reconfiguration time. In such scenarios, optimizing the phase-shift design helps mitigate performance degradation while ensuring specific QoS requirements. Moreover, the proposed algorithm has been tested through experimental evaluations, which demonstrate that it also performs effectively in practice.
Variability-Driven User-Story Generation using LLM and Triadic Concept Analysis
Authors: Alexandre Bazin, Alain Gutierrez, Marianne Huchard, Pierre Martin, Yulin, Zhang
Date: 2025-04-11 00:00:00
Tasks: Large Language Model, Story Generation

A widely used Agile practice for requirements is to produce a set of user stories (also called ``agile product backlog''), which roughly includes a list of pairs (role, feature), where the role handles the feature for a certain purpose. In the context of Software Product Lines, the requirements for a family of similar systems is thus a family of user-story sets, one per system, leading to a 3-dimensional dataset composed of sets of triples (system, role, feature). In this paper, we combine Triadic Concept Analysis (TCA) and Large Language Model (LLM) prompting to suggest the user-story set required to develop a new system relying on the variability logic of an existing system family. This process consists in 1) computing 3-dimensional variability expressed as a set of TCA implications, 2) providing the designer with intelligible design options, 3) capturing the designer's selection of options, 4) proposing a first user-story set corresponding to this selection, 5) consolidating its validity according to the implications identified in step 1, while completing it if necessary, and 6) leveraging LLM to have a more comprehensive website. This process is evaluated with a dataset comprising the user-story sets of 67 similar-purpose websites.