pwc-archive/papers-with-abstracts huggingface dataset:
Feature-Preserving Mesh Decimation for Normal Integration
Authors: Moritz Heep, Sven Behnke, Eduard Zell
Date: 2025-04-01 00:00:00
Proceeding: CVPR 2025 1

Normal integration reconstructs 3D surfaces from normal maps obtained e.g. by photometric stereo. These normal maps capture surface details down to the pixel level but require large computational resources for integration at high resolutions. In this work, we replace the dense pixel grid with a sparse anisotropic triangle mesh prior to normal integration. We adapt the triangle mesh to the local geometry in the case of complex surface structures and remove oversampling from flat featureless regions. For high-resolution images, the resulting compression reduces normal integration runtimes from hours to minutes while maintaining high surface accuracy. Our main contribution is the derivation of the well-known quadric error measure from mesh decimation for screen space applications and its combination with optimal Delaunay triangulation.
CBIL: Collective Behavior Imitation Learning for Fish from Real Videos
Authors: Yifan Wu, Zhiyang Dou, Yuko Ishiwaka, Shun Ogawa, Yuke Lou, Wenping Wang, Lingjie Liu, Taku Komura
Date: 2025-03-31 00:00:00
Tasks: Imitation Learning, Representation Learning

Reproducing realistic collective behaviors presents a captivating yet formidable challenge. Traditional rule-based methods rely on hand-crafted principles, limiting motion diversity and realism in generated collective behaviors. Recent imitation learning methods learn from data but often require ground truth motion trajectories and struggle with authenticity, especially in high-density groups with erratic movements. In this paper, we present a scalable approach, Collective Behavior Imitation Learning (CBIL), for learning fish schooling behavior directly from videos, without relying on captured motion trajectories. Our method first leverages Video Representation Learning, where a Masked Video AutoEncoder (MVAE) extracts implicit states from video inputs in a self-supervised manner. The MVAE effectively maps 2D observations to implicit states that are compact and expressive for following the imitation learning stage. Then, we propose a novel adversarial imitation learning method to effectively capture complex movements of the schools of fish, allowing for efficient imitation of the distribution for motion patterns measured in the latent space. It also incorporates bio-inspired rewards alongside priors to regularize and stabilize training. Once trained, CBIL can be used for various animation tasks with the learned collective motion priors. We further show its effectiveness across different species. Finally, we demonstrate the application of our system in detecting abnormal fish behavior from in-the-wild videos.
Mesh Compression with Quantized Neural Displacement Fields
Authors: Sai Karthikey Pentapati, Gregoire Phillips, Alan C. Bovik
Date: 2025-03-28 00:00:00

Implicit neural representations (INRs) have been successfully used to compress a variety of 3D surface representations such as Signed Distance Functions (SDFs), voxel grids, and also other forms of structured data such as images, videos, and audio. However, these methods have been limited in their application to unstructured data such as 3D meshes and point clouds. This work presents a simple yet effective method that extends the usage of INRs to compress 3D triangle meshes. Our method encodes a displacement field that refines the coarse version of the 3D mesh surface to be compressed using a small neural network. Once trained, the neural network weights occupy much lower memory than the displacement field or the original surface. We show that our method is capable of preserving intricate geometric textures and demonstrates state-of-the-art performance for compression ratios ranging from 4x to 380x.
LayerCraft: Enhancing Text-to-Image Generation with CoT Reasoning and Layered Object Integration
Authors: Yuyao Zhang, Jinghao Li, Yu-Wing Tai
Date: 2025-03-25 00:00:00
Tasks: Image Generation, Object, Prompt Engineering, Scene Generation, Text to Image Generation, Text-to-Image Generation

Text-to-image (T2I) generation has made remarkable progress, yet existing systems still lack intuitive control over spatial composition, object consistency, and multi-step editing. We present $\textbf{LayerCraft}$, a modular framework that uses large language models (LLMs) as autonomous agents to orchestrate structured, layered image generation and editing. LayerCraft supports two key capabilities: (1) $\textit{structured generation}$ from simple prompts via chain-of-thought (CoT) reasoning, enabling it to decompose scenes, reason about object placement, and guide composition in a controllable, interpretable manner; and (2) $\textit{layered object integration}$, allowing users to insert and customize objects -- such as characters or props -- across diverse images or scenes while preserving identity, context, and style. The system comprises a coordinator agent, the $\textbf{ChainArchitect}$ for CoT-driven layout planning, and the $\textbf{Object Integration Network (OIN)}$ for seamless image editing using off-the-shelf T2I models without retraining. Through applications like batch collage editing and narrative scene generation, LayerCraft empowers non-experts to iteratively design, customize, and refine visual content with minimal manual effort. Code will be released at https://github.com/PeterYYZhang/LayerCraft.
Matching, Unanticipated Experiences, Divorce, Flirting, Rematching, Etc
Authors: Burkhard C. Schipper, Tina Danting Zhang
Date: 2025-04-02 00:00:00
Tasks: Blocking, Friction

We study dynamic decentralized two-sided matching in which players may encounter unanticipated experiences. As they become aware of these experiences, they may change their preferences over players on the other side of the market. Consequently, they may get ``divorced'' and rematch again with other agents, which may lead to further unanticipated experiences etc. A matching is stable if there is absence of pairwise common belief in blocking. Stable matchings can be destabilized by unanticipated experiences. Yet, we show that there exist self-confirming outcomes that are stable and do not lead to further unanticipated experiences. We introduce a natural decentralized matching process that, at each period assigns probability $1 - \varepsilon$ to the satisfaction of a mutual optimal blocking pair (if it exists) and picks any optimal blocking pair otherwise. The parameter $\varepsilon$ is interpreted as a friction of the matching market. We show that for any decentralized matching process, frictions are necessary for convergence to stability even without unawareness. Our process converges to self-confirming stable outcomes. Further, we allow for bilateral communication/flirting that changes the awareness and say that a matching is flirt-proof stable if there is absence of communication leading to pairwise common belief in blocking. We show that our natural decentralized matching process converges to flirt-proof self-confirming outcomes.
Off-Policy Evaluation for Sequential Persuasion Process with Unobserved Confounding
Authors: Nishanth Venkatesh S., Heeseung Bang, Andreas A. Malikopoulos
Date: 2025-04-01 00:00:00
Tasks: Decision Making, Off-policy evaluation, Sequential Decision Making

In this paper, we expand the Bayesian persuasion framework to account for unobserved confounding variables in sender-receiver interactions. While traditional models assume that belief updates follow Bayesian principles, real-world scenarios often involve hidden variables that impact the receiver's belief formation and decision-making. We conceptualize this as a sequential decision-making problem, where the sender and receiver interact over multiple rounds. In each round, the sender communicates with the receiver, who also interacts with the environment. Crucially, the receiver's belief update is affected by an unobserved confounding variable. By reformulating this scenario as a Partially Observable Markov Decision Process (POMDP), we capture the sender's incomplete information regarding both the dynamics of the receiver's beliefs and the unobserved confounder. We prove that finding an optimal observation-based policy in this POMDP is equivalent to solving for an optimal signaling strategy in the original persuasion framework. Furthermore, we demonstrate how this reformulation facilitates the application of proximal learning for off-policy evaluation in the persuasion process. This advancement enables the sender to evaluate alternative signaling strategies using only observational data from a behavioral policy, thus eliminating the necessity for costly new experiments.
Remember, but also, Forget: Bridging Myopic and Perfect Recall Fairness with Past-Discounting
Authors: Ashwin Kumar, William Yeoh
Date: 2025-04-01 00:00:00
Tasks: Decision Making, Fairness, Sequential Decision Making

Dynamic resource allocation in multi-agent settings often requires balancing efficiency with fairness over time--a challenge inadequately addressed by conventional, myopic fairness measures. Motivated by behavioral insights that human judgments of fairness evolve with temporal distance, we introduce a novel framework for temporal fairness that incorporates past-discounting mechanisms. By applying a tunable discount factor to historical utilities, our approach interpolates between instantaneous and perfect-recall fairness, thereby capturing both immediate outcomes and long-term equity considerations. Beyond aligning more closely with human perceptions of fairness, this past-discounting method ensures that the augmented state space remains bounded, significantly improving computational tractability in sequential decision-making settings. We detail the formulation of discounted-recall fairness in both additive and averaged utility contexts, illustrate its benefits through practical examples, and discuss its implications for designing balanced, scalable resource allocation strategies.
A Parametric Model for Near-Optimal Online Synthesis with Robust Reach-Avoid Guarantees
Authors: Mario Gleirscher, Philip Hönnecke
Date: 2025-04-01 00:00:00

Objective: To obtain explainable guarantees in the online synthesis of optimal controllers for high-integrity cyber-physical systems, we re-investigate the use of exhaustive search as an alternative to reinforcement learning. Approach: We model an application scenario as a hybrid game automaton, enabling the synthesis of robustly correct and near-optimal controllers online without prior training. For modal synthesis, we employ discretised games solved via scope-adaptive and step-pre-shielded discrete dynamic programming. Evaluation: In a simulation-based experiment, we apply our approach to an autonomous aerial vehicle scenario. Contribution: We propose a parametric system model and a parametric online synthesis.
Chaos and noise in evolutionary game dynamics
Authors: Maria Alejandra Ramirez, George Datseris, Arne Traulsen
Date: 2025-03-28 00:00:00

Evolutionary game theory has traditionally employed deterministic models to describe population dynamics. These models, due to their inherent nonlinearities, can exhibit deterministic chaos, where population fluctuations follow complex, aperiodic patterns. Recently, the focus has shifted towards stochastic models, quantifying fixation probabilities and analysing systems with constants of motion. Yet, the role of stochastic effects in systems with chaotic dynamics remains largely unexplored within evolutionary game theory. This study addresses how demographic noise -- arising from probabilistic birth and death events -- impacts chaotic dynamics in finite populations. We show that despite stochasticity, large populations retain a signature of chaotic dynamics, as evidenced by comparing a chaotic deterministic system with its stochastic counterpart. More concretely, the strange attractor observed in the deterministic model is qualitatively recovered in the stochastic model, where the term deterministic chaos loses its meaning. We employ tools from nonlinear dynamics to quantify how the population size influences the dynamics. We observe that for small populations, stochasticity dominates, overshadowing deterministic selection effects. However, as population size increases, the dynamics increasingly reflect the underlying chaotic structure. This resilience to demographic noise can be essential for maintaining diversity in populations, even in non-equilibrium dynamics. Overall, our results broaden our understanding of population dynamics, and revisit the boundaries between chaos and noise, showing how they maintain structure when considering finite populations in systems that are chaotic in the deterministic limit.
Advancing AI-Scientist Understanding: Making LLM Think Like a Physicist with Interpretable Reasoning
Authors: Yinggan Xu, Hana Kimlee, Yijia Xiao, Di Luo
Date: 2025-04-02 00:00:00
Tasks: scientific discovery

Large Language Models (LLMs) are playing an expanding role in physics research by enhancing reasoning, symbolic manipulation, and numerical computation. However, ensuring the reliability and interpretability of their outputs remains a significant challenge. In our framework, we conceptualize the collaboration between AI and human scientists as a dynamic interplay among three modules: the reasoning module, the interpretation module, and the AI-scientist interaction module. Recognizing that effective physics reasoning demands rigorous logical consistency, quantitative precision, and deep integration with established theoretical models, we introduce the interpretation module to improve the understanding of AI-generated outputs, which is not previously explored in the literature. This module comprises multiple specialized agents, including summarizers, model builders, UI builders, and testers, which collaboratively structure LLM outputs within a physically grounded framework, by constructing a more interpretable science model. A case study demonstrates that our approach enhances transparency, facilitates validation, and strengthens AI-augmented reasoning in scientific discovery.
A novel gesture interaction control method for rehabilitation lower extremity exoskeleton
Authors: Shuang Qiu, Zhongcai Pei, Chen Wang, Jing Zhang, Zhiyong Tang
Date: 2025-04-02 00:00:00
Tasks: Depth Estimation

With the rapid development of Rehabilitation Lower Extremity Robotic Exoskeletons (RLEEX) technology, significant advancements have been made in Human-Robot Interaction (HRI) methods. These include traditional physical HRI methods that are easily recognizable and various bio-electrical signal-based HRI methods that can visualize and predict actions. However, most of these HRI methods are contact-based, facing challenges such as operational complexity, sensitivity to interference, risks associated with implantable devices, and, most importantly, limitations in comfort. These challenges render the interaction less intuitive and natural, which can negatively impact patient motivation for rehabilitation. To address these issues, this paper proposes a novel non-contact gesture interaction control method for RLEEX, based on RGB monocular camera depth estimation. This method integrates three key steps: detecting keypoints, recognizing gestures, and assessing distance, thereby applying gesture information and augmented reality triggering technology to control gait movements of RLEEX. Results indicate that this approach provides a feasible solution to the problems of poor comfort, low reliability, and high latency in HRI for RLEEX platforms. Specifically, it achieves a gesture-controlled exoskeleton motion accuracy of 94.11\% and an average system response time of 0.615 seconds through non-contact HRI. The proposed non-contact HRI method represents a pioneering advancement in control interactions for RLEEX, paving the way for further exploration and development in this field.
TransforMerger: Transformer-based Voice-Gesture Fusion for Robust Human-Robot Communication
Authors: Petr Vanc, Karla Stepanova
Date: 2025-04-02 00:00:00
Tasks: Language Modeling, Language Modelling, Object, Scene Understanding, Sentence

As human-robot collaboration advances, natural and flexible communication methods are essential for effective robot control. Traditional methods relying on a single modality or rigid rules struggle with noisy or misaligned data as well as with object descriptions that do not perfectly fit the predefined object names (e.g. 'Pick that red object'). We introduce TransforMerger, a transformer-based reasoning model that infers a structured action command for robotic manipulation based on fused voice and gesture inputs. Our approach merges multimodal data into a single unified sentence, which is then processed by the language model. We employ probabilistic embeddings to handle uncertainty and we integrate contextual scene understanding to resolve ambiguous references (e.g., gestures pointing to multiple objects or vague verbal cues like "this"). We evaluate TransforMerger in simulated and real-world experiments, demonstrating its robustness to noise, misalignment, and missing information. Our results show that TransforMerger outperforms deterministic baselines, especially in scenarios requiring more contextual knowledge, enabling more robust and flexible human-robot communication. Code and datasets are available at: http://imitrob.ciirc.cvut.cz/publications/transformerger.
Reasoning LLMs for User-Aware Multimodal Conversational Agents
Authors: Hamed Rahimi, Jeanne Cattoni, Meriem Beghili, Mouad Abrini, Mahdi Khoramshahi, Maribel Pino, Mohamed Chetouani
Date: 2025-04-02 00:00:00
Tasks: RAG, Retrieval-augmented Generation, Visual Question Answering (VQA)

Personalization in social robotics is critical for fostering effective human-robot interactions, yet systems often face the cold start problem, where initial user preferences or characteristics are unavailable. This paper proposes a novel framework called USER-LLM R1 for a user-aware conversational agent that addresses this challenge through dynamic user profiling and model initiation. Our approach integrates chain-of-thought (CoT) reasoning models to iteratively infer user preferences and vision-language models (VLMs) to initialize user profiles from multimodal inputs, enabling personalized interactions from the first encounter. Leveraging a Retrieval-Augmented Generation (RAG) architecture, the system dynamically refines user representations within an inherent CoT process, ensuring contextually relevant and adaptive responses. Evaluations on the ElderlyTech-VQA Bench demonstrate significant improvements in ROUGE-1 (+23.2%), ROUGE-2 (+0.6%), and ROUGE-L (+8%) F1 scores over state-of-the-art baselines, with ablation studies underscoring the impact of reasoning model size on performance. Human evaluations further validate the framework's efficacy, particularly for elderly users, where tailored responses enhance engagement and trust. Ethical considerations, including privacy preservation and bias mitigation, are rigorously discussed and addressed to ensure responsible deployment.
Virtual Reality and Artificial Intelligence as Psychological Countermeasures in Space and Other Isolated and Confined Environments: A Scoping Review
Authors: Jennifer Sharp, Joshua Kelson, Daryl South, Anthony Saliba, Muhammad Ashad Kabir
Date: 2025-04-02 00:00:00

Spaceflight is an isolated and confined environment (ICE) that exposes astronauts to psychological hazards, such as stress, danger, and monotony. Virtual reality (VR) and artificial intelligence (AI) technologies can serve as psychological countermeasures as they can digitally simulate immersive environments, interactive companions, and therapeutic experiences. Our study employs a scoping literature review approach to identify what is currently known about the use and effectiveness of VR and AI-based interventions as psychological countermeasures to improve mood or emotional states in adults in space or other ICEs. Additionally, this review aimed to identify gaps in the knowledge base and whether a systematic review with meta-analysis was warranted. The review included studies where the intervention was used or intended for use in space or other extraterrestrial environments (ICE). Our search strategy yielded 19 studies from 3390 records across seven major databases. All studies focused on VR-based interventions, with no eligible AI-based intervention studies found. VR interventions were found to be effective for relaxation and improving mood, emergency training, as an interactive communication platform, for comparing interior designs, and for enhancing exercise. There were improvements for measures of mood and emotion\n (e.g., anxiety and stress); however, user preferences varied, and some instances of cybersickness were reported. A systematic review with meta-analysis is not recommended due to the heterogeneity of results. There is significant scope for further research into the use of VR for a wider range of mood and emotion variables using standardised assessment instruments. Additionally, the potential application of AI as a psychological countermeasure warrants further investigation.
Interactive Sketchpad: A Multimodal Tutoring System for Collaborative, Visual Problem-Solving
Authors: Steven-Shine Chen, JiMin Lee, Paul Pu Liang
Date: 2025-02-12 00:00:00
Tasks: Math, multimodal interaction

Humans have long relied on visual aids like sketches and diagrams to support reasoning and problem-solving. Visual tools, like auxiliary lines in geometry or graphs in calculus, are essential for understanding complex ideas. However, many tutoring systems remain text-based, providing feedback only through natural language. Leveraging recent advances in Large Multimodal Models (LMMs), this paper introduces Interactive Sketchpad, a tutoring system that combines language-based explanations with interactive visualizations to enhance learning. Built on a pre-trained LMM, Interactive Sketchpad is fine-tuned to provide step-by-step guidance in both text and visuals, enabling natural multimodal interaction with the student. Accurate and robust diagrams are generated by incorporating code execution into the reasoning process. User studies conducted on math problems such as geometry, calculus, and trigonometry demonstrate that Interactive Sketchpad leads to improved task comprehension, problem-solving accuracy, and engagement levels, highlighting its potential for transforming educational technologies. All code is available at: https://stevenshinechen.github.io/interactivesketchpad/.