pwc-archive/papers-with-abstracts huggingface dataset:
State Combinatorial Generalization In Decision Making With Conditional Diffusion Models
Authors: Xintong Duan, Yutong He, Fahim Tajwar, Wen-Tse Chen, Ruslan Salakhutdinov, Jeff Schneider
Date: 2025-01-22 00:00:00
Tasks: Decision Making, Reinforcement Learning (RL), Zero-shot Generalization

Many real-world decision-making problems are combinatorial in nature, where states (e.g., surrounding traffic of a self-driving car) can be seen as a combination of basic elements (e.g., pedestrians, trees, and other cars). Due to combinatorial complexity, observing all combinations of basic elements in the training set is infeasible, which leads to an essential yet understudied problem of zero-shot generalization to states that are unseen combinations of previously seen elements. In this work, we first formalize this problem and then demonstrate how existing value-based reinforcement learning (RL) algorithms struggle due to unreliable value predictions in unseen states. We argue that this problem cannot be addressed with exploration alone, but requires more expressive and generalizable models. We demonstrate that behavior cloning with a conditioned diffusion model trained on expert trajectory generalizes better to states formed by new combinations of seen elements than traditional RL methods. Through experiments in maze, driving, and multiagent environments, we show that conditioned diffusion models outperform traditional RL techniques and highlight the broad applicability of our problem formulation.
MLPs at the EOC: Spectrum of the NTK
Authors: Dávid Terjék, Diego González-Sánchez
Date: 2025-01-22 00:00:00

We study the properties of the Neural Tangent Kernel (NTK) $\overset{\scriptscriptstyle\infty}{K} : \mathbb{R}^{m_0} \times \mathbb{R}^{m_0} \to \mathbb{R}^{m_l \times m_l}$ corresponding to infinitely wide $l$-layer Multilayer Perceptrons (MLPs) taking inputs from $\mathbb{R}^{m_0}$ to outputs in $\mathbb{R}^{m_l}$ equipped with activation functions $\phi(s) = a s + b \vert s \vert$ for some $a,b \in \mathbb{R}$ and initialized at the Edge Of Chaos (EOC). We find that the entries $\overset{\scriptscriptstyle\infty}{K}(x_1,x_2)$ can be approximated by the inverses of the cosine distances of the activations corresponding to $x_1$ and $x_2$ increasingly better as the depth $l$ increases. By quantifying these inverse cosine distances and the spectrum of the matrix containing them, we obtain tight spectral bounds for the NTK matrix $\overset{\scriptscriptstyle\infty}{K} = [\frac{1}{n} \overset{\scriptscriptstyle\infty}{K}(x_{i_1},x_{i_2}) : i_1, i_2 \in [1:n]]$ over a dataset $\{x_1,\cdots,x_n\} \subset \mathbb{R}^{m_0}$, transferred from the inverse cosine distance matrix via our approximation result. Our results show that $\Delta_\phi = \frac{b^2}{a^2+b^2}$ determines the rate at which the condition number of the NTK matrix converges to its limit as depth increases, implying in particular that the absolute value ($\Delta_\phi=1$) is better than the ReLU ($\Delta_\phi=\frac{1}{2}$) in this regard.
A Comprehensive Social Bias Audit of Contrastive Vision Language Models
Authors: Zahraa Al Sahili, Ioannis Patras, Matthew Purver
Date: 2025-01-22 00:00:00
Tasks: Diversity, Fairness, Lightweight Deployment

In the domain of text-to-image generative models, biases inherent in training datasets often propagate into generated content, posing significant ethical challenges, particularly in socially sensitive contexts. We introduce FairCoT, a novel framework that enhances fairness in text-to-image models through Chain-of-Thought (CoT) reasoning within multimodal generative large language models. FairCoT employs iterative CoT refinement to systematically mitigate biases, and dynamically adjusts textual prompts in real time, ensuring diverse and equitable representation in generated images. By integrating iterative reasoning processes, FairCoT addresses the limitations of zero-shot CoT in sensitive scenarios, balancing creativity with ethical responsibility. Experimental evaluations across popular text-to-image systems--including DALL-E and various Stable Diffusion variants--demonstrate that FairCoT significantly enhances fairness and diversity without sacrificing image quality or semantic fidelity. By combining robust reasoning, lightweight deployment, and extensibility to multiple models, FairCoT represents a promising step toward more socially responsible and transparent AI-driven content generation.
Enhancing Multi-Attribute Fairness in Healthcare Predictive Modeling
Authors: Xiaoyang Wang, Christopher C. Yang
Date: 2025-01-22 00:00:00
Tasks: Attribute, Fairness

Artificial intelligence (AI) systems in healthcare have demonstrated remarkable potential to improve patient outcomes. However, if not designed with fairness in mind, they also carry the risks of perpetuating or exacerbating existing health disparities. Although numerous fairness-enhancing techniques have been proposed, most focus on a single sensitive attribute and neglect the broader impact that optimizing fairness for one attribute may have on the fairness of other sensitive attributes. In this work, we introduce a novel approach to multi-attribute fairness optimization in healthcare AI, tackling fairness concerns across multiple demographic attributes concurrently. Our method follows a two-phase approach: initially optimizing for predictive performance, followed by fine-tuning to achieve fairness across multiple sensitive attributes. We develop our proposed method using two strategies, sequential and simultaneous. Our results show a significant reduction in Equalized Odds Disparity (EOD) for multiple attributes, while maintaining high predictive accuracy. Notably, we demonstrate that single-attribute fairness methods can inadvertently increase disparities in non-targeted attributes whereas simultaneous multi-attribute optimization achieves more balanced fairness improvements across all attributes. These findings highlight the importance of comprehensive fairness strategies in healthcare AI and offer promising directions for future research in this critical area.
S-LoRA: Scalable Low-Rank Adaptation for Class Incremental Learning
Authors: Yichen Wu, Hongming Piao, Long-Kai Huang, Renzhen Wang, Wanhua Li, Hanspeter Pfister, Deyu Meng, Kede Ma, Ying WEI
Date: 2025-01-22 00:00:00
Tasks: class-incremental learning, Class Incremental Learning, Continual Learning, Incremental Learning

Continual Learning with foundation models has recently emerged as a promising approach to harnessing the power of pre-trained models for sequential tasks. Existing prompt-based methods generally use a gating mechanism to select relevant prompts aligned with the test query for further processing. However, the success of these methods largely depends on the precision of the gating mechanism, which becomes less scalable with additional computational overhead as tasks increases. To overcome these issues, we propose a Scalable Low-Rank Adaptation (S-LoRA) method for CL (in particular class incremental learning), which incrementally decouples the learning of the direction and magnitude of LoRA parameters. S-LoRA supports efficient inference by employing the last-stage trained model for direct testing without a gating process. Our theoretical and empirical analysis demonstrates that S-LoRA tends to follow a low-loss trajectory that converges to an overlapped low-loss region, resulting in an excellent stability-plasticity trade-off in CL. Furthermore, based on our findings, we develop variants of S-LoRA with further improved scalability. Extensive experiments across multiple CL benchmarks and various foundation models consistently validate the effectiveness of S-LoRA.
Topological constraints on self-organisation in locally interacting systems
Authors: Francesco Sacco, Dalton A R Sakthivadivel, Michael Levin
Date: 2025-01-22 00:00:00

All intelligence is collective intelligence, in the sense that it is made of parts which must align with respect to system-level goals. Understanding the dynamics which facilitate or limit navigation of problem spaces by aligned parts thus impacts many fields ranging across life sciences and engineering. To that end, consider a system on the vertices of a planar graph, with pairwise interactions prescribed by the edges of the graph. Such systems can sometimes exhibit long-range order, distinguishing one phase of macroscopic behaviour from another. In networks of interacting systems we may view spontaneous ordering as a form of self-organisation, modelling neural and basal forms of cognition. Here, we discuss necessary conditions on the topology of the graph for an ordered phase to exist, with an eye towards finding constraints on the ability of a system with local interactions to maintain an ordered target state. By studying the scaling of free energy under the formation of domain walls in three model systems -- the Potts model, autoregressive models, and hierarchical networks -- we show how the combinatorics of interactions on a graph prevent or allow spontaneous ordering. As an application we are able to analyse why multiscale systems like those prevalent in biology are capable of organising into complex patterns, whereas rudimentary language models are challenged by long sequences of outputs.
Efficient Implementation of LinearUCB through Algorithmic Improvements and Vector Computing Acceleration for Embedded Learning Systems
Authors: Marco Angioli, Marcello Barbirotta, Abdallah Cheikh, Antonio Mastrandrea, Francesco Menichelli, Mauro Olivieri
Date: 2025-01-22 00:00:00
Tasks: Decision Making, Edge-computing, Multi-Armed Bandits

As the Internet of Things expands, embedding Artificial Intelligence algorithms in resource-constrained devices has become increasingly important to enable real-time, autonomous decision-making without relying on centralized cloud servers. However, implementing and executing complex algorithms in embedded devices poses significant challenges due to limited computational power, memory, and energy resources. This paper presents algorithmic and hardware techniques to efficiently implement two LinearUCB Contextual Bandits algorithms on resource-constrained embedded devices. Algorithmic modifications based on the Sherman-Morrison-Woodbury formula streamline model complexity, while vector acceleration is harnessed to speed up matrix operations. We analyze the impact of each optimization individually and then combine them in a two-pronged strategy. The results show notable improvements in execution time and energy consumption, demonstrating the effectiveness of combining algorithmic and hardware optimizations to enhance learning models for edge computing environments with low-power and real-time requirements.
Forecasting of Bitcoin Prices Using Hashrate Features: Wavelet and Deep Stacking Approach
Authors: Ramin Mousa, Meysam Afrookhteh, Hooman Khaloo, Amir Ali Bengari, Gholamreza Heidary
Date: 2025-01-22 00:00:00
Tasks: feature selection

Digital currencies have become popular in the last decade due to their non-dependency and decentralized nature. The price of these currencies has seen a lot of fluctuations at times, which has increased the need for prediction. As their most popular, Bitcoin(BTC) has become a research hotspot. The main challenge and trend of digital currencies, especially BTC, is price fluctuations, which require studying the basic price prediction model. This research presents a classification and regression model based on stack deep learning that uses a wavelet to remove noise to predict movements and prices of BTC at different time intervals. The proposed model based on the stacking technique uses models based on deep learning, especially neural networks and transformers, for one, seven, thirty and ninety-day forecasting. Three feature selection models, Chi2, RFE and Embedded, were also applied to the data in the pre-processing stage. The classification model achieved 63\% accuracy for predicting the next day and 64\%, 67\% and 82\% for predicting the seventh, thirty and ninety days, respectively. For daily price forecasting, the percentage error was reduced to 0.58, while the error ranged from 2.72\% to 2.85\% for seven- to ninety-day horizons. These results show that the proposed model performed better than other models in the literature.
UniRestore: Unified Perceptual and Task-Oriented Image Restoration Model Using Diffusion Prior
Authors: I-Hsiang Chen, Wei-Ting Chen, Yu-Wei Liu, Yuan-Chun Chiang, Sy-Yen Kuo, Ming-Hsuan Yang
Date: 2025-01-22 00:00:00
Proceeding: CVPR 2025 1
Tasks: Image Restoration, Unified Image Restoration

Image restoration aims to recover content from inputs degraded by various factors, such as adverse weather, blur, and noise. Perceptual Image Restoration (PIR) methods improve visual quality but often do not support downstream tasks effectively. On the other hand, Task-oriented Image Restoration (TIR) methods focus on enhancing image utility for high-level vision tasks, sometimes compromising visual quality. This paper introduces UniRestore, a unified image restoration model that bridges the gap between PIR and TIR by using a diffusion prior. The diffusion prior is designed to generate images that align with human visual quality preferences, but these images are often unsuitable for TIR scenarios. To solve this limitation, UniRestore utilizes encoder features from an autoencoder to adapt the diffusion prior to specific tasks. We propose a Complementary Feature Restoration Module (CFRM) to reconstruct degraded encoder features and a Task Feature Adapter (TFA) module to facilitate adaptive feature fusion in the decoder. This design allows UniRestore to optimize images for both human perception and downstream task requirements, addressing discrepancies between visual quality and functional needs. Integrating these modules also enhances UniRestore's adapability and efficiency across diverse tasks. Extensive expertments demonstrate the superior performance of UniRestore in both PIR and TIR scenarios.
Graph Representation Learning with Diffusion Generative Models
Authors: Daniel Wesego
Date: 2025-01-22 00:00:00
Tasks: Denoising, Graph Representation Learning, Representation Learning

Diffusion models have established themselves as state-of-the-art generative models across various data modalities, including images and videos, due to their ability to accurately approximate complex data distributions. Unlike traditional generative approaches such as VAEs and GANs, diffusion models employ a progressive denoising process that transforms noise into meaningful data over multiple iterative steps. This gradual approach enhances their expressiveness and generation quality. Not only that, diffusion models have also been shown to extract meaningful representations from data while learning to generate samples. Despite their success, the application of diffusion models to graph-structured data remains relatively unexplored, primarily due to the discrete nature of graphs, which necessitates discrete diffusion processes distinct from the continuous methods used in other domains. In this work, we leverage the representational capabilities of diffusion models to learn meaningful embeddings for graph data. By training a discrete diffusion model within an autoencoder framework, we enable both effective autoencoding and representation learning tailored to the unique characteristics of graph-structured data. We only need the encoder at the end to extract representations. Our approach demonstrates the potential of discrete diffusion models to be used for graph representation learning.
A Learnt Half-Quadratic Splitting-Based Algorithm for Fast and High-Quality Industrial Cone-beam CT Reconstruction
Authors: Aniket Pramanik, Singanallur V. Venkatakrishnan, Obaidullah Rahman, Amirkoushyar Ziabari
Date: 2025-01-21 00:00:00
Tasks: CT Reconstruction

Industrial X-ray cone-beam CT (XCT) scanners are widely used for scientific imaging and non-destructive characterization. Industrial CBCT scanners use large detectors containing millions of pixels and the subsequent 3D reconstructions can be of the order of billions of voxels. In order to obtain high-quality reconstruction when using typical analytic algorithms, the scan involves collecting a large number of projections/views which results in large measurement times - limiting the utility of the technique. Model-based iterative reconstruction (MBIR) algorithms can produce high-quality reconstructions from fast sparse-view CT scans, but are computationally expensive and hence are avoided in practice. Single-step deep-learning (DL) based methods have demonstrated that it is possible to obtain fast and high-quality reconstructions from sparse-view data but they do not generalize well to out-of-distribution scenarios. In this work, we propose a half-quadratic splitting-based algorithm that uses convolutional neural networks (CNN) in order to obtain high-quality reconstructions from large sparse-view cone-beam CT (CBCT) measurements while overcoming the challenges with typical approaches. The algorithm alternates between the application of a CNN and a conjugate gradient (CG) step enforcing data-consistency (DC). The proposed method outperforms other methods on the publicly available Walnuts data-set.
Dual-level Adaptive Incongruity-enhanced Model for Multimodal Sarcasm Detection
Authors: Qiaofeng Wu, Wenlong Fang, Weiyu Zhong, Fenghuan Li, Yun Xue, Bo Chen
Date: 2025-01-07 00:00:00
Proceeding: Neurocomputing 2025 1
Tasks: Contrastive Learning, Sarcasm Detection

Multimodal sarcasm detection leverages multimodal information, such as image, text, etc. to identify special instances whose superficial emotional expression is contrary to the actual emotion. Existing methods primarily focused on the incongruity between text and image information for sarcasm detection. Existing sarcasm methods in which the tendency of image encoders to encode similar images into similar vectors, and the introduction of noise in graph-level feature extraction due to negative correlations caused by the accumulation of GAT layers and the lack of representations for non-neighboring nodes. To address these limitations, we propose a Dual-Level Adaptive Incongruity-Enhanced Model (DAIE) to extract the incongruity between the text and image at both token and graph levels. At the token level, we bolster token-level contrastive learning with patch-based reconstructed image to capture common and specific features of images, thereby amplifying incongruities between text and images. At the graph level, we introduce adaptive graph contrast learning, coupled with negative pair similarity weights, to refine the feature representation of the model’s textual and visual graph nodes, while also enhancing the information exchange among neighboring nodes. We conduct experiments using a publicly available sarcasm detection dataset. The results demonstrate the effectiveness of our method, outperforming several state-of-the-art approaches by 3.33% and 4.34% on accuracy and F1 score, respectively.
Reducing Size Bias in Epidemic Network Modelling
Authors: Neha Bansal, Katerina Kaouri, Thomas E. Woolley
Date: 2025-01-22 00:00:00

Epidemiological models help policymakers mitigate disease spread by predicting transmission metrics based on disease dynamics and contact networks. Calibrating these models requires representative network sampling. We investigate the Random Walk (RW) and Metropolis-Hastings Random Walk (MHRW) algorithms for three network types: Erd\H{o}s-R\'enyi (ER), Small-world (SW), and Scale-free (SF). Disease transmission is simulated using a stochastic susceptible-infected-recovered (SIR) framework. For ER and SW networks, RW overestimates infected individuals and secondary infections by $25\%$ due to size bias, favouring highly connected nodes. MHRW, though more computationally intensive, reduces size bias and provides more representative samples. For time-to-infection, both algorithms provide representative estimates. However, neither algorithm samples SF networks representatively, exhibiting significant variability. Furthermore, removing duplicate sample nodes reduces MHRW's accuracy across three network types. We apply both algorithms to a cattle movement network of 46,512 farms combining ER, SW, and SF features. RW overestimates infected farms by about $100\%$ and secondary infections by over $900\%$, reflecting significant size bias, while MHRW estimates align within $1\%$ of the cattle network values. RW underestimates time-to-infection by about $40\%$, while MHRW overestimates it by $10\%$. Accuracy, again, deteriorates when duplicates nodes are removed. Our findings guide algorithm selection and intervention strategies based on network structure and disease severity; RW's conservative estimates suit high-mortality, fast-spreading epidemics, while MHRW enables more precise interventions for slower epidemics.
Takagi–Sugeno fuzzy model based robust dissipative control for uncertain flexible spacecraft with saturated time-delay input
Authors: Shidong Xu, Guanghui Sunn, Weichao Sun
Date: 2016-10-12 00:00:00
Proceeding: 期刊 2016 10

In this paper, the problem of robust dissipative control is investigated for uncertain flexible spacecraft
 based onTakagi–Sugeno (T–S) fuzzy model with saturated time-delay input. Different from most existing
 strategies, T–S fuzzy approximation approach is used to model the nonlinear dynamics of flexible
 spacecraft. Simultaneously, the physical constraints of system, like input delay, input saturation, and
 parameter uncertainties, are also taken care of in the fuzzy model. By employing Lyapunov–Krasovskii
 method and convex optimization technique, a novel robust controller is proposed to implement rest-to
rest attitude maneuver for flexible spacecraft, and the guaranteed dissipative performance enables the
 uncertain closed-loop system to reject the influence of elastic vibrations and external disturbances. Fi
nally, an illustrative design example integrated with simulation results are provided to confirm the
 applicability and merits of the developed control strategy.
iServe: An Intent-based Serving System for LLMs
Authors: Dimitrios Liakopoulos, Tianrui Hu, Prasoon Sinha, Neeraja J. Yadwadkar
Date: 2025-01-08 00:00:00
Tasks: GPU

Large Language Models (LLMs) are becoming ubiquitous across industries, where applications demand they fulfill diverse user intents. However, developers currently face the challenge of manually exploring numerous deployment configurations - combinations of parallelism and compression techniques that impact resource usage, latency, cost, and accuracy - to meet these intents. Assessing the impact of these configurations on user metrics requires extensive, costly profiling for each model. Existing approaches avoid this expense by using fixed, static configurations, but this often leads to sub-optimal performance and higher costs. Moreover, none of these solutions dynamically adapt to changing user intents to balance latency and cost, effectively. We present iServe, an automated, intent-based system for distributed LLM inference. Instead of manually selecting deployment configurations, developers simply specify their intent - such as minimizing latency, reducing cost, or meeting specific targets for either. iServe introduces fingerprints, lightweight representations of LLMs, to efficiently estimate how different configurations impact latency and memory usage. Based on these insights and GPU availability, iServe dynamically selects the optimal configuration to align with the user's intent. For various LLMs and query arrival rates, iServe best meets user intents compared to state-of-the-art systems by reducing latency by 77.62% and SLO violations by 7.09x while improving GPU throughput by 4.72x. Moreover, iServe's fingerprint-based profiling reduces profiling cost by 6.05x (GPU-hours) compared to baselines.