pwc-archive/papers-with-abstracts huggingface dataset:
EmoTalkingGaussian: Continuous Emotion-conditioned Talking Head Synthesis
Authors: Junuk Cha, Seongro Yoon, Valeriya Strizhkova, Francois Bremond, Seungryul Baek
Date: 2025-02-02 00:00:00
Tasks: Self-Supervised Learning, SSIM, text-to-speech, Text to Speech

3D Gaussian splatting-based talking head synthesis has recently gained attention for its ability to render high-fidelity images with real-time inference speed. However, since it is typically trained on only a short video that lacks the diversity in facial emotions, the resultant talking heads struggle to represent a wide range of emotions. To address this issue, we propose a lip-aligned emotional face generator and leverage it to train our EmoTalkingGaussian model. It is able to manipulate facial emotions conditioned on continuous emotion values (i.e., valence and arousal); while retaining synchronization of lip movements with input audio. Additionally, to achieve the accurate lip synchronization for in-the-wild audio, we introduce a self-supervised learning method that leverages a text-to-speech network and a visual-audio synchronization network. We experiment our EmoTalkingGaussian on publicly available videos and have obtained better results than state-of-the-arts in terms of image quality (measured in PSNR, SSIM, LPIPS), emotion expression (measured in V-RMSE, A-RMSE, V-SA, A-SA, Emotion Accuracy), and lip synchronization (measured in LMD, Sync-E, Sync-C), respectively.
RM-PoT: Reformulating Mathematical Problems and Solving via Program of Thoughts
Authors: Yu Zhang, Shujun Peng, Nengwu Wu, Xinhan Lin, Yang Hu, Jie Tang
Date: 2025-02-18 00:00:00
Tasks: Few-Shot Learning

Recently, substantial advancements have been made in training language models to carry out step-by-step reasoning for solving intricate numerical reasoning tasks. Beyond the methods used to solve these problems, the structure and formulation of the problems themselves also play a crucial role in determining the performance of large language models. We observe that even small changes in the surface form of mathematical problems can have a profound impact on both the answer distribution and solve rate. This highlights the vulnerability of LLMs to surface-level variations, revealing its limited robustness when reasoning through complex problems. In this paper, we propose RM-PoT, a three-stage framework that integrates problem reformulation (RM), code-aided reasoning (PoT), and domain-aware few-shot learning to address these limitations. Our approach first reformulates the input problem into diverse surface forms to reduce structural bias, then retrieves five semantically aligned examples from a pre-constructed domain-specific question bank to provide contextual guidance, and finally generates executable Python code for precise computation.
Boost, Disentangle, and Customize: A Robust System2-to-System1 Pipeline for Code Generation
Authors: Kounianhua Du, Hanjing Wang, Jianxing Liu, Jizheng Chen, Xinyi Dai, Yasheng Wang, Ruiming Tang, Yong Yu, Jun Wang, Weinan Zhang
Date: 2025-02-18 00:00:00
Tasks: Code Generation

Large language models (LLMs) have demonstrated remarkable capabilities in various domains, particularly in system 1 tasks, yet the intricacies of their problem-solving mechanisms in system 2 tasks are not sufficiently explored. Recent research on System2-to-System1 methods surge, exploring the System 2 reasoning knowledge via inference-time computation and compressing the explored knowledge into System 1 process. In this paper, we focus on code generation, which is a representative System 2 task, and identify two primary challenges: (1) the complex hidden reasoning processes and (2) the heterogeneous data distributions that complicate the exploration and training of robust LLM solvers. To tackle these issues, we propose a novel BDC framework that explores insightful System 2 knowledge of LLMs using a MC-Tree-Of-Agents algorithm with mutual \textbf{B}oosting, \textbf{D}isentangles the heterogeneous training data for composable LoRA-experts, and obtain \textbf{C}ustomized problem solver for each data instance with an input-aware hypernetwork to weight over the LoRA-experts, offering effectiveness, flexibility, and robustness. This framework leverages multiple LLMs through mutual verification and boosting, integrated into a Monte-Carlo Tree Search process enhanced by reflection-based pruning and refinement. Additionally, we introduce the DisenLora algorithm, which clusters heterogeneous data to fine-tune LLMs into composable Lora experts, enabling the adaptive generation of customized problem solvers through an input-aware hypernetwork. This work lays the groundwork for advancing LLM capabilities in complex reasoning tasks, offering a novel System2-to-System1 solution.
MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluation
Authors: Yutong Wang, Pengliang Ji, Chaoqun Yang, Kaixin Li, Ming Hu, Jiaoyang Li, Guillaume Sartoretti
Date: 2025-02-18 00:00:00
Tasks: global-optimization, Large Language Model

The LLM-as-a-Judge paradigm shows promise for evaluating generative content but lacks reliability in reasoning-intensive scenarios, such as programming. Inspired by recent advances in reasoning models and shifts in scaling laws, we pioneer bringing test-time computation into LLM-as-a-Judge, proposing MCTS-Judge, a resource-efficient, System-2 thinking framework for code correctness evaluation. MCTS-Judge leverages Monte Carlo Tree Search (MCTS) to decompose problems into simpler, multi-perspective evaluations. Through a node-selection strategy that combines self-assessment based on historical actions in the current trajectory and the Upper Confidence Bound for Trees based on prior rollouts, MCTS-Judge balances global optimization and refinement of the current trajectory. We further designed a high-precision, unit-test-level reward mechanism to encourage the Large Language Model (LLM) to perform line-by-line analysis. Extensive experiments on three benchmarks and five LLMs demonstrate the effectiveness of MCTS-Judge, which improves the base model's accuracy from 41% to 80%, surpassing the o1-series models with 3x fewer tokens. Further evaluations validate the superiority of its reasoning trajectory in logic, analytics, thoroughness, and overall quality, while revealing the test-time scaling law of the LLM-as-a-Judge paradigm.
Independence Tests for Language Models
Authors: Sally Zhu, Ahmed Ahmed, Rohith Kuditipudi, Percy Liang
Date: 2025-02-17 00:00:00

We consider the following problem: given the weights of two models, can we test whether they were trained independently -- i.e., from independent random initializations? We consider two settings: constrained and unconstrained. In the constrained setting, we make assumptions about model architecture and training and propose a family of statistical tests that yield exact p-values with respect to the null hypothesis that the models are trained from independent random initializations. These p-values are valid regardless of the composition of either model's training data; we compute them by simulating exchangeable copies of each model under our assumptions and comparing various similarity measures of weights and activations between the original two models versus these copies. We report the p-values from these tests on pairs of 21 open-weight models (210 total pairs) and correctly identify all pairs of non-independent models. Our tests remain effective even if one model was fine-tuned for many tokens. In the unconstrained setting, where we make no assumptions about training procedures, can change model architecture, and allow for adversarial evasion attacks, the previous tests no longer work. Instead, we propose a new test which matches hidden activations between two models, and which is robust to adversarial transformations and to changes in model architecture. The test can also do localized testing: identifying specific non-independent components of models. Though we no longer obtain exact p-values from this, empirically we find it behaves as one and reliably identifies non-independent models. Notably, we can use the test to identify specific parts of one model that are derived from another (e.g., how Llama 3.1-8B was pruned to initialize Llama 3.2-3B, or shared layers between Mistral-7B and StripedHyena-7B), and it is even robust to retraining individual layers of either model from scratch.
PAR-AdvGAN: Improving Adversarial Attack Capability with Progressive Auto-Regression AdvGAN
Authors: Jiayu Zhang, Zhiyu Zhu, Xinyi Wang, Silin Liao, Zhibo Jin, Flora D. Salim, Huaming Chen
Date: 2025-02-16 00:00:00
Tasks: Adversarial Attack, regression

Deep neural networks have demonstrated remarkable performance across various domains. However, they are vulnerable to adversarial examples, which can lead to erroneous predictions. Generative Adversarial Networks (GANs) can leverage the generators and discriminators model to quickly produce high-quality adversarial examples. Since both modules train in a competitive and simultaneous manner, GAN-based algorithms like AdvGAN can generate adversarial examples with better transferability compared to traditional methods. However, the generation of perturbations is usually limited to a single iteration, preventing these examples from fully exploiting the potential of the methods. To tackle this issue, we introduce a novel approach named Progressive Auto-Regression AdvGAN (PAR-AdvGAN). It incorporates an auto-regressive iteration mechanism within a progressive generation network to craft adversarial examples with enhanced attack capability. We thoroughly evaluate our PAR-AdvGAN method with a large-scale experiment, demonstrating its superior performance over various state-of-the-art black-box adversarial attacks, as well as the original AdvGAN.Moreover, PAR-AdvGAN significantly accelerates the adversarial example generation, i.e., achieving the speeds of up to 335.5 frames per second on Inception-v3 model, outperforming the gradient-based transferable attack algorithms. Our code is available at: https://anonymous.4open.science/r/PAR-01BF/
To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models
Authors: Zihao Zhu, Hongbao Zhang, Ruotong Wang, Ke Xu, Siwei Lyu, Baoyuan Wu
Date: 2025-02-16 00:00:00
Tasks: Adversarial Attack, Backdoor Attack

Large Reasoning Models (LRMs) are designed to solve complex tasks by generating explicit reasoning traces before producing final answers. However, we reveal a critical vulnerability in LRMs -- termed Unthinking Vulnerability -- wherein the thinking process can be bypassed by manipulating special delimiter tokens. It is empirically demonstrated to be widespread across mainstream LRMs, posing both a significant risk and potential utility, depending on how it is exploited. In this paper, we systematically investigate this vulnerability from both malicious and beneficial perspectives. On the malicious side, we introduce Breaking of Thought (BoT), a novel attack that enables adversaries to bypass the thinking process of LRMs, thereby compromising their reliability and availability. We present two variants of BoT: a training-based version that injects backdoor during the fine-tuning stage, and a training-free version based on adversarial attack during the inference stage. As a potential defense, we propose thinking recovery alignment to partially mitigate the vulnerability. On the beneficial side, we introduce Monitoring of Thought (MoT), a plug-and-play framework that allows model owners to enhance efficiency and safety. It is implemented by leveraging the same vulnerability to dynamically terminate redundant or risky reasoning through external monitoring. Extensive experiments show that BoT poses a significant threat to reasoning reliability, while MoT provides a practical solution for preventing overthinking and jailbreaking. Our findings expose an inherent flaw in current LRM architectures and underscore the need for more robust reasoning systems in the future.
Recursive generalized type-2 fuzzy radial basis function neural networks for joint position estimation and adaptive EMG-based impedance control of lower limb exoskeletons
Authors: Kianoush Aqabakee, Farzaneh Abdollahi, Afshin Taghvaeipour, Mohammad-R Akbarzadeh-T
Date: 2025-02-01 00:00:00
Proceeding: Biomedical Signal Processing and Control 2025 2
Tasks: Electromyography (EMG), GPU, Position, Reinforcement Learning (RL)

Electromyography (EMG) is a common method to estimate users’ intended motion in exoskeleton robots. Yet, it is highly susceptible to noise, and the process is complex due to its dynamic nature. While deep neural networks (DNNs) have been utilized to address these challenges, their high computational demands have necessitated costly GPU hardware and posed barriers to portable medical applications. This study introduces a novel approach employing a recurrent structure based on radial basis function neural networks (RBFNN) alongside a type-2 fuzzy inference system to mitigate computational costs while handling increased complexity. Integrating RBFNN and fuzzy inference enhances the model’s ability to process data with fewer parameters and reduced mathematical computations, enabling real-time practical solutions. An impedance controller manages robot joints, while a fuzzy system determines the control parameters. Most fuzzy model parameters are selected via a data-driven approach facilitated by a reinforcement learning (RL) algorithm acting as an expert and augmented by adaptive rules within the control cycle for the remaining parameters. Patients with mobility disorders can use video games to enhance the effectiveness of their rehabilitation. Also, video games can aid patients by customizing the controller’s parameters to suit their needs. A hybrid learning framework that combines supervised learning and RL has been developed and tested to determine fuzzy model parameters through a car driving task. The framework achieves precision and generalization similar to convolutional neural networks with similar normalized root mean square error (NRMSE) and correlation coefficient but fewer computational costs and floating-point operations per second.
Wearanize+: A Multimodal Dataset for Evaluating Wearable Technologies in Sleep Research
Authors: Niloy Sikder, Lieuwe Verkaar, Anastasiya Paltarzhytskaya, Selin Acan, Elena Krugliakova, Yevgenia Rosenblum, Matthias Krauledat, Martin Dresler, and Paul Zerr
Date: 2025-01-30 00:00:00
Proceeding: Center for Open Science 2025 1
Tasks: Classification, EEG

Sleep research heavily relies on Polysomnography (PSG) recordings to assess sleep architecture. While effective, this method is time-consuming and requires substantial resources and labor. Modern wearable devices provide a promising alternative for sleep monitoring as they are easy to wear and maintain. However, these devices are constrained by a limited number of channels and comparatively lower data quality, which often leads to unreliable outcomes derived from partial readings. To address this, we propose using multiple wearable devices and combining their outputs to acquire reliable sleep data. However, the feasibility of this approach must be rigorously tested before being relied upon to supplant PSG in scientific studies. To facilitate this, we have curated a dataset with concurrent full PSG and wearable device recordings of overnight sleep sessions. This dataset, named Wearanize+, includes data from 130 participants (one night each) using three different wearable devices: Zmax headband, Empatica E4 wristband, and ActivPAL leg patch, alongside full-scale PSG recorded with SomnoScreen Plus and Mentalab Explore Pro. It also includes questionnaires, such as PSQI, MADRE, and PHQ-9, providing information on participants’ sleep, dreams, and overall health. This paper documents the setup, data collection, and data preprocessing steps of the Wearanize+ project, serving as a guide for using the resultant dataset. Our objectives with the dataset include developing machine learning models that can derive PSG-grade sleep stages from wearable devices’ data and exploring alternative data modalities for sleep scoring, especially when EEG signals are excessively noisy. Given its potential to support the testing of a wide range of hypotheses, we have made the dataset openly available to the research community.
KU-HAR: An open dataset for heterogeneous human activity recognition
Authors: Niloy Sikder, Abdullah-Al Nahid
Date: 2021-03-17 00:00:00
Proceeding: Pattern Recognition Letters 2021 3
Tasks: Activity Recognition, Classification, Ensemble Learning, Human Activity Recognition

In Artificial Intelligence, Human Activity Recognition (HAR) refers to the capability of machines to identify various activities performed by the users. The knowledge acquired from these recognition systems is integrated into many applications where the associated device uses it to identify actions or gestures and performs predefined tasks in response. HAR requires a large quantity of meticulously collected action data of diverse nature to fuel its learning algorithms. This paper aims to introduce a new set of HAR data collected from 90 participants who are 18 to 34 years old. The constructed dataset is named KU-HAR, and it contains 1945 raw activity samples that belong to 18 different classes. We used built-in smartphone sensors (accelerometer and gyroscope) to collect these HAR samples. Apart from the original (raw) time-domain samples, the dataset contains 20,750 subsamples (extracted from them) provided separately, each containing 3 seconds of data of the corresponding activity. Some classification results have been provided as well to propound the quality of the proposed dataset. The acquired results show that Random Forest (RF), an ensemble learning algorithm, can classify the subsamples with almost 90% accuracy. This dataset will enable smartphones and other smart devices to identify new activities and help researchers to design more delicate models based on practical HAR data.
Flare-Free Vision: Empowering Uformer with Depth Insights
Authors: Yousef Kotp, Marwan Torki
Date: 2024-04-01 00:00:00
Proceeding: ICASSP 2024 4
Tasks: Depth Estimation, Flare Removal, Image Restoration

Image flare is a common problem that occurs when a camera lens is pointed at a strong light source. It can manifest as ghosting, blooming, or other artifacts that can degrade the image quality. We propose a novel deep learning approach for flare removal that uses a combination of depth estimation and image restoration. We use a Dense Vision Transformer to estimate the depth of the scene. This depth map is then concatenated to the input image, which is then fed into a Uformer, a general U-shaped transformer for image restoration. Our proposed method demonstrates state-of-the-art performance on the Flare7K++ test dataset, demonstrating its effectiveness in removing flare artifacts from images. Our approach also demonstrates robustness and generalization to real-world images with various types of flare. We believe that our work opens up new possibilities for using depth information for image restoration. The code is available on GitHub
Multi-view Instance Attention Fusion Network for classification
Authors: Jinxing Lia, Chuhao Zhoua, Xiaoqiang Jic, Mu Lia, Guangming Lua, Yong Xua, B, David Zhang
Date: 2023-08-22 00:00:00
Proceeding: 101974 2023 8
Tasks: Classification, MULTI-VIEW LEARNING

Multi-view learning for classification has achieved a remarkable performance compared with the single-view based methods. Inspired by the instance based learning which directly regards the instance as the prior and well preserves the valuable information in different instances, a Multi-view Instance Attention Fusion Network (MvIAFN) is proposed to efficiently exploit the correlation across both instances and views. Specifically, a small number of instances from different views are first sampled as the set of templates. Given an additional instance and based on the similarities between it and the selected templates, it can be re-presented by following an attention strategy. Thanks for this strategy, the given instance is capable of preserving the additional information from the selected instances, achieving the purpose of extracting the instance-correlation. Additionally, for each sample, we not only perform the instance attention in each single view but also get the attention across multiple views, allowing us to further fuse them to obtain the fused attention for each view. Experimental results on datasets substantiate the effectiveness of our proposed method compared with state-of-the-arts.
CLR-GAN: Improving GANs Stability and Quality via Consistent Latent Representation and Reconstruction
Authors: Shengke Sun, Ziqian Luan, Zhanshan Zhao, Shijie Luo and Shuzhen Han
Date: 2024-09-30 00:00:00
Proceeding: ECCV 2024 9
Tasks: Image Generation

Generative Adversarial Networks(GANs) have received considerable attention due to its outstanding ability to generate images. However, training a GAN is hard since the game between the Generator(G) and the Discriminator(D) is unfair. Towards making the competition fairer, we propose a new perspective of training GANs, named Consistent Latent Representation and Reconstruction(CLR-GAN). In this paradigm, we treat the G and D as an inverse process, the discriminator has an additional task to restore the pre-defined latent code while the generator also needs to reconstruct the real input, thus obtaining a relationship between the latent space of G and the out-features of D. Based on this prior, we can put D and G on an equal position during training using a new criterion. Experimental results on various datasets and architectures prove our paradigm can make GANs more stable and generate better quality images(31.22% gain of FID on CIFAR10 and 39.5% on AFHQ-Cat, respectively). We hope that the proposed perspective can inspire researchers to explore different ways of viewing GANs training, rather than being limited to a two-player game. The code is publicly available at https://github.com/Petecheco/CLR-GAN.
Investigating and Extending Homans' Social Exchange Theory with Large Language Model based Agents
Authors: Lei Wang, Zheqing Zhang, Xu Chen
Date: 2025-02-18 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model

Homans' Social Exchange Theory (SET) is widely recognized as a basic framework for understanding the formation and emergence of human civilizations and social structures. In social science, this theory is typically studied based on simple simulation experiments or real-world human studies, both of which either lack realism or are too expensive to control. In artificial intelligence, recent advances in large language models (LLMs) have shown promising capabilities in simulating human behaviors. Inspired by these insights, we adopt an interdisciplinary research perspective and propose using LLM-based agents to study Homans' SET. Specifically, we construct a virtual society composed of three LLM agents and have them engage in a social exchange game to observe their behaviors. Through extensive experiments, we found that Homans' SET is well validated in our agent society, demonstrating the consistency between the agent and human behaviors. Building on this foundation, we intentionally alter the settings of the agent society to extend the traditional Homans' SET, making it more comprehensive and detailed. To the best of our knowledge, this paper marks the first step in studying Homans' SET with LLM-based agents. More importantly, it introduces a novel and feasible research paradigm that bridges the fields of social science and computer science through LLM-based agents. Code is available at https://github.com/Paitesanshi/SET.