pwc-archive/papers-with-abstracts huggingface dataset:
CoMoGaussian: Continuous Motion-Aware Gaussian Splatting from Motion-Blurred Images
Authors: Jungho Lee, Donghyeong Kim, Dogyoon Lee, Suhwan Cho, Minhyeok Lee, Wonjoon Lee, Taeoh Kim, Dongyoon Wee, Sangyoun Lee
Date: 2025-03-07 00:00:00
Tasks: 3DGS, 3D Scene Reconstruction

3D Gaussian Splatting (3DGS) has gained significant attention for their high-quality novel view rendering, motivating research to address real-world challenges. A critical issue is the camera motion blur caused by movement during exposure, which hinders accurate 3D scene reconstruction. In this study, we propose CoMoGaussian, a Continuous Motion-Aware Gaussian Splatting that reconstructs precise 3D scenes from motion-blurred images while maintaining real-time rendering speed. Considering the complex motion patterns inherent in real-world camera movements, we predict continuous camera trajectories using neural ordinary differential equations (ODEs). To ensure accurate modeling, we employ rigid body transformations, preserving the shape and size of the object but rely on the discrete integration of sampled frames. To better approximate the continuous nature of motion blur, we introduce a continuous motion refinement (CMR) transformation that refines rigid transformations by incorporating additional learnable parameters. By revisiting fundamental camera theory and leveraging advanced neural ODE techniques, we achieve precise modeling of continuous camera trajectories, leading to improved reconstruction accuracy. Extensive experiments demonstrate state-of-the-art performance both quantitatively and qualitatively on benchmark datasets, which include a wide range of motion blur scenarios, from moderate to extreme blur.
Fake It To Make It: Virtual Multiviews to Enhance Monocular Indoor Semantic Scene Completion
Authors: Anith Selvakumar, Manasa Bharadwaj
Date: 2025-03-07 00:00:00
Tasks: Novel View Synthesis

Monocular Indoor Semantic Scene Completion (SSC) aims to reconstruct a 3D semantic occupancy map from a single RGB image of an indoor scene, inferring spatial layout and object categories from 2D image cues. The challenge of this task arises from the depth, scale, and shape ambiguities that emerge when transforming a 2D image into 3D space, particularly within the complex and often heavily occluded environments of indoor scenes. Current SSC methods often struggle with these ambiguities, resulting in distorted or missing object representations. To overcome these limitations, we introduce an innovative approach that leverages novel view synthesis and multiview fusion. Specifically, we demonstrate how virtual cameras can be placed around the scene to emulate multiview inputs that enhance contextual scene information. We also introduce a Multiview Fusion Adaptor (MVFA) to effectively combine the multiview 3D scene predictions into a unified 3D semantic occupancy map. Finally, we identify and study the inherent limitation of generative techniques when applied to SSC, specifically the Novelty-Consistency tradeoff. Our system, GenFuSE, demonstrates IoU score improvements of up to 2.8% for Scene Completion and 4.9% for Semantic Scene Completion when integrated with existing SSC networks on the NYUv2 dataset. This work introduces GenFuSE as a standard framework for advancing monocular SSC with synthesized inputs.
A Modular Pipeline for 3D Object Tracking Using RGB Cameras
Authors: Lars Bredereke, Yale Hartmann, Tanja Schultz
Date: 2025-03-06 00:00:00
Tasks: 3D Object Tracking, Object Tracking

Object tracking is a key challenge of computer vision with various applications that all require different architectures. Most tracking systems have limitations such as constraining all movement to a 2D plane and they often track only one object. In this paper, we present a new modular pipeline that calculates 3D trajectories of multiple objects. It is adaptable to various settings where multiple time-synced and stationary cameras record moving objects, using off the shelf webcams. Our pipeline was tested on the Table Setting Dataset, where participants are recorded with various sensors as they set a table with tableware objects. We need to track these manipulated objects, using 6 rgb webcams. Challenges include: Detecting small objects in 9.874.699 camera frames, determining camera poses, discriminating between nearby and overlapping objects, temporary occlusions, and finally calculating a 3D trajectory using the right subset of an average of 11.12.456 pixel coordinates per 3-minute trial. We implement a robust pipeline that results in accurate trajectories with covariance of x,y,z-position as a confidence metric. It deals dynamically with appearing and disappearing objects, instantiating new Extended Kalman Filters. It scales to hundreds of table-setting trials with very little human annotation input, even with the camera poses of each trial unknown. The code is available at https://github.com/LarsBredereke/object_tracking
Geometry-Constrained Monocular Scale Estimation Using Semantic Segmentation for Dynamic Scenes
Authors: HUI ZHANG, Zhiyang Wu, Qianqian Shangguan, Kang An
Date: 2025-03-06 00:00:00
Tasks: Autonomous Driving, Computational Efficiency, Monocular Visual Odometry, Motion Estimation, Semantic Segmentation, Visual Localization, Visual Odometry

Monocular visual localization plays a pivotal role in advanced driver assistance systems and autonomous driving by estimating a vehicle's ego-motion from a single pinhole camera. Nevertheless, conventional monocular visual odometry encoun-ters challenges in scale estimation due to the absence of depth information during projection. Previous methodologies, whether rooted in physical constraints or deep learning paradigms, con-tend with issues related to computational complexity and the management of dynamic objects. This study extends our prior research, presenting innovative strategies for ego-motion estima-tion and the selection of ground points. Striving for a nuanced equilibrium between computational efficiency and precision, we propose a hybrid method that leverages the SegNeXt model for real-time applications, encompassing both ego-motion estimation and ground point selection. Our methodology incorporates dy-namic object masks to eliminate unstable features and employs ground plane masks for meticulous triangulation. Furthermore, we exploit Geometry-constraint to delineate road regions for scale recovery. The integration of this approach with the mo-nocular version of ORB-SLAM3 culminates in the accurate esti-mation of a road model, a pivotal component in our scale recov-ery process. Rigorous experiments, conducted on the KITTI da-taset, systematically compare our method with existing monocu-lar visual odometry algorithms and contemporary scale recovery methodologies. The results undeniably confirm the superior ef-fectiveness of our approach, surpassing state-of-the-art visual odometry algorithms. Our source code is available at https://git hub.com/bFr0zNq/MVOSegScale.
Adversarial Policy Optimization for Offline Preference-based Reinforcement Learning
Authors: Hyungkyu Kang, Min-hwan Oh
Date: 2025-03-07 00:00:00
Tasks: continuous-control, Continuous Control, reinforcement-learning, Reinforcement Learning

In this paper, we study offline preference-based reinforcement learning (PbRL), where learning is based on pre-collected preference feedback over pairs of trajectories. While offline PbRL has demonstrated remarkable empirical success, existing theoretical approaches face challenges in ensuring conservatism under uncertainty, requiring computationally intractable confidence set constructions. We address this limitation by proposing Adversarial Preference-based Policy Optimization (APPO), a computationally efficient algorithm for offline PbRL that guarantees sample complexity bounds without relying on explicit confidence sets. By framing PbRL as a two-player game between a policy and a model, our approach enforces conservatism in a tractable manner. Using standard assumptions on function approximation and bounded trajectory concentrability, we derive a sample complexity bound. To our knowledge, APPO is the first offline PbRL algorithm to offer both statistical efficiency and practical applicability. Experimental results on continuous control tasks demonstrate that APPO effectively learns from complex datasets, showing comparable performance with existing state-of-the-art methods.
Policy Constraint by Only Support Constraint for Offline Reinforcement Learning
Authors: Yunkai Gao, Jiaming Guo, Fan Wu, Rui Zhang
Date: 2025-03-07 00:00:00
Tasks: Offline RL, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Offline reinforcement learning (RL) aims to optimize a policy by using pre-collected datasets, to maximize cumulative rewards. However, offline reinforcement learning suffers challenges due to the distributional shift between the learned and behavior policies, leading to errors when computing Q-values for out-of-distribution (OOD) actions. To mitigate this issue, policy constraint methods aim to constrain the learned policy's distribution with the distribution of the behavior policy or confine action selection within the support of the behavior policy. However, current policy constraint methods tend to exhibit excessive conservatism, hindering the policy from further surpassing the behavior policy's performance. In this work, we present Only Support Constraint (OSC) which is derived from maximizing the total probability of learned policy in the support of behavior policy, to address the conservatism of policy constraint. OSC presents a regularization term that only restricts policies to the support without imposing extra constraints on actions within the support. Additionally, to fully harness the performance of the new policy constraints, OSC utilizes a diffusion model to effectively characterize the support of behavior policies. Experimental evaluations across a variety of offline RL benchmarks demonstrate that OSC significantly enhances performance, alleviating the challenges associated with distributional shifts and mitigating conservatism of policy constraints. Code is available at https://github.com/MoreanP/OSC.
Multi-Task Reinforcement Learning Enables Parameter Scaling
Authors: Reginald McLean, Evangelos Chatzarouals, Jordan Terry, Isaac Woungang, Nariman Farsad, Pablo Samuel Castro
Date: 2025-03-07 00:00:00
Tasks: Diversity, reinforcement-learning, Reinforcement Learning

Multi-task reinforcement learning (MTRL) aims to endow a single agent with the ability to perform well on multiple tasks. Recent works have focused on developing novel sophisticated architectures to improve performance, often resulting in larger models; it is unclear, however, whether the performance gains are a consequence of the architecture design itself or the extra parameters. We argue that gains are mostly due to scale by demonstrating that naively scaling up a simple MTRL baseline to match parameter counts outperforms the more sophisticated architectures, and these gains benefit most from scaling the critic over the actor. Additionally, we explore the training stability advantages that come with task diversity, demonstrating that increasing the number of tasks can help mitigate plasticity loss. Our findings suggest that MTRL's simultaneous training across multiple tasks provides a natural framework for beneficial parameter scaling in reinforcement learning, challenging the need for complex architectural innovations.
On a Connection Between Imitation Learning and RLHF
Authors: Teng Xiao, Yige Yuan, Mingxiao Li, Zhengyu Chen, Vasant G Honavar
Date: 2025-03-07 00:00:00
Tasks: Imitation Learning

This work studies the alignment of large language models with preference data from an imitation learning perspective. We establish a close theoretical connection between reinforcement learning from human feedback RLHF and imitation learning (IL), revealing that RLHF implicitly performs imitation learning on the preference data distribution. Building on this connection, we propose DIL, a principled framework that directly optimizes the imitation learning objective. DIL provides a unified imitation learning perspective on alignment, encompassing existing alignment algorithms as special cases while naturally introducing new variants. By bridging IL and RLHF, DIL offers new insights into alignment with RLHF. Extensive experiments demonstrate that DIL outperforms existing methods on various challenging benchmarks.
Energy-Weighted Flow Matching for Offline Reinforcement Learning
Authors: Shiyuan Zhang, Weitong Zhang, Quanquan Gu
Date: 2025-03-06 00:00:00
Tasks: Offline RL, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper investigates energy guidance in generative modeling, where the target distribution is defined as $q(\mathbf x) \propto p(\mathbf x)\exp(-\beta \mathcal E(\mathbf x))$, with $p(\mathbf x)$ being the data distribution and $\mathcal E(\mathcal x)$ as the energy function. To comply with energy guidance, existing methods often require auxiliary procedures to learn intermediate guidance during the diffusion process. To overcome this limitation, we explore energy-guided flow matching, a generalized form of the diffusion process. We introduce energy-weighted flow matching (EFM), a method that directly learns the energy-guided flow without the need for auxiliary models. Theoretical analysis shows that energy-weighted flow matching accurately captures the guided flow. Additionally, we extend this methodology to energy-weighted diffusion models and apply it to offline reinforcement learning (RL) by proposing the Q-weighted Iterative Policy Optimization (QIPO). Empirically, we demonstrate that the proposed QIPO algorithm improves performance in offline RL tasks. Notably, our algorithm is the first energy-guided diffusion model that operates independently of auxiliary models and the first exact energy-guided flow matching model in the literature.
VQEL: Enabling Self-Developed Symbolic Language in Agents through Vector Quantization in Emergent Language Games
Authors: Mohammad Mahdi Samiei Paqaleh, Mahdieh Soleymani Baghshah
Date: 2025-03-06 00:00:00
Tasks: Quantization

In the field of emergent language, efforts have traditionally focused on developing communication protocols through interactions between agents in referential games. However, the aspect of internal language learning, where language serves not only as a communicative tool with others but also as a means for individual thinking, self-reflection, and problem-solving remains underexplored. Developing a language through self-play, without another agent's involvement, poses a unique challenge. It requires an agent to craft symbolic representations and train them using direct gradient methods. The challenge here is that if an agent attempts to learn symbolic representations through self-play using conventional modeling and techniques such as REINFORCE, the solution will offer no advantage over previous multi-agent approaches. We introduce VQEL, a novel method that incorporates Vector Quantization into the agents' architecture, enabling them to autonomously invent and develop discrete symbolic representations in a self-play referential game. Following the self-play phase, agents can enhance their language through reinforcement learning and interactions with other agents in the mutual-play phase. Our experiments across various datasets demonstrate that VQEL not only outperforms the traditional REINFORCE method but also benefits from improved control and reduced susceptibility to collapse, thanks to the incorporation of vector quantization.
Sample-Optimal Agnostic Boosting with Unlabeled Data
Authors: Udaya Ghai, Karan Singh
Date: 2025-03-06 00:00:00

Boosting provides a practical and provably effective framework for constructing accurate learning algorithms from inaccurate rules of thumb. It extends the promise of sample-efficient learning to settings where direct Empirical Risk Minimization (ERM) may not be implementable efficiently. In the realizable setting, boosting is known to offer this computational reprieve without compromising on sample efficiency. However, in the agnostic case, existing boosting algorithms fall short of achieving the optimal sample complexity. This paper highlights an unexpected and previously unexplored avenue of improvement: unlabeled samples. We design a computationally efficient agnostic boosting algorithm that matches the sample complexity of ERM, given polynomially many additional unlabeled samples. In fact, we show that the total number of samples needed, unlabeled and labeled inclusive, is never more than that for the best known agnostic boosting algorithm -- so this result is never worse -- while only a vanishing fraction of these need to be labeled for the algorithm to succeed. This is particularly fortuitous for learning-theoretic applications of agnostic boosting, which often take place in the distribution-specific setting, where unlabeled samples can be availed for free. We detail other applications of this result in reinforcement learning.
Leveraging Large Language Models For Scalable Vector Graphics Processing: A Review
Authors: Boris Malashenko, Ivan Jarsky, Valeria Efimova
Date: 2025-03-06 00:00:00
Tasks: Vector Graphics

In recent years, rapid advances in computer vision have significantly improved the processing and generation of raster images. However, vector graphics, which is essential in digital design, due to its scalability and ease of editing, have been relatively understudied. Traditional vectorization techniques, which are often used in vector generation, suffer from long processing times and excessive output complexity, limiting their usability in practical applications. The advent of large language models (LLMs) has opened new possibilities for the generation, editing, and analysis of vector graphics, particularly in the SVG format, which is inherently text-based and well-suited for integration with LLMs. This paper provides a systematic review of existing LLM-based approaches for SVG processing, categorizing them into three main tasks: generation, editing, and understanding. We observe notable models such as IconShop, StrokeNUWA, and StarVector, highlighting their strengths and limitations. Furthermore, we analyze benchmark datasets designed for assessing SVG-related tasks, including SVGEditBench, VGBench, and SGP-Bench, and conduct a series of experiments to evaluate various LLMs in these domains. Our results demonstrate that for vector graphics reasoning-enhanced models outperform standard LLMs, particularly in generation and understanding tasks. Furthermore, our findings underscore the need to develop more diverse and richly annotated datasets to further improve LLM capabilities in vector graphics tasks.
Semantic tradeoff for heterogeneous graph embedding
Authors: Yunfei He, Dengcheng Yan, Yiwen Zhang, Qiang He, and Yun Yang, Senior Member, IEEE
Date: 2023-06-01 00:00:00
Proceeding: IEEE Transactions on Computational Social Systems 2023 6
Tasks: Graph Attention, Graph Embedding, Graph Neural Network

Abstract— Recently, many data mining models based on heterogeneous graph (HG) have emerged. Among these, HG embedding is an important and indispensable process. However, the existing HG embedding models usually use graph neural network to learn embeddings separately on different meta-paths, ignoring the fact that different meta-paths contain not only their unique semantics but also related semantics. This may result in semantic overlap or irrelevance, which needs to be a feasible and effective tradeoff for high-quality HG embedding, yet studies thereof have rarely been reported. In this article, we propose semantic tradeoff HG embedding (STHGE) by first introducing the Hilbert–Schmidt independence criterion (HSIC) as restriction. The main idea of STHGE is to regard semantic tradeoff as independence tradeoff (or correlation) between different meta-path spaces. Specifically, we first transform the original features of nodes into different meta-path feature spaces with HSIC restriction between them. Then, we use graph attention network to learn the embeddings of nodes on different meta-paths with HSIC restrictions. Finally, we concatenate the embeddings on different meta-paths to perform prediction. Experimental results on three heterogeneous datasets not only demonstrate the effectiveness of STHGE but also demonstrate that STHGE can achieve a new semantic tradeoff between different meta-paths. Furthermore, we demonstrate the robustness of STHGE.
Context-aware Biases for Length Extrapolation
Authors: Ali Veisi, Amir Mansourian
Date: 2025-03-11 00:00:00
Tasks: Decoder

Transformers' ability to generalize to longer sequences than they have been trained on, known as length extrapolation, degrades as sequence length increases. Most of Relative Positional Encoding (RPE) methods address this problem by either adding constant linear biases or learning general biases, lacking the ability to specialize for different sequences. In this work, inspired by ALiBi, we propose Context-aware Biases for Length Extrapolation (Cable), that learns token-specific biases for each head in decoder-based transformers. Cable learns adaptive, context-aware biases, overcoming the limitations of fixed patterns by adding dynamic biases specific to each token in the sequence. Results show that when tested on a sequence length of 1024, a GPT-3 Medium (334M parameters) with our positional encoding, trained on a sequence length of 512, achieves better perplexity (-0.65) than a similar network with sinusoidal positional encoding trained on a sequence length of 1024. This is achieved with 48% lower memory usage, and only 3.5% higher training time. Furthermore, our method notably improves the extrapolation ability of existing RPE methods on the Edu-FineWeb10B and WikiText-103 datasets. Code is available at: https://github.com/axiomlab/Cable
SR-HGN: semantic- and relation-aware heterogeneous graph neural network
Authors: Zehong Wang a, B, Donghua Yu a, ∗, Qi Li a, Shigen Shen c, Shuang Yao d
Date: 2023-08-15 00:00:00
Proceeding: Expert Systems with Applications 2023 8
Tasks: Graph Neural Network, Node Classification, Node Clustering, Relation

Graph Neural Networks (GNNs) have received considerable attention in recent years due to their unique ability to model both topologies and semantics in the graphs. In this paper, we explore the use of GNNs in learning low-dimensional node representations in Heterogeneous Information Networks (HINs), which retain rich semantic information across multiple types of nodes and relations. Existing methods for HINs generally focus on modeling heterogeneity at either the node level or the relation level, but not both. As a result, these methods often fall short of optimal performance. To address this issue, we propose a novel Semanticand Relation-aware Heterogeneous Graph neural Network, dubbed SR-HGN, which jointly incorporates rich semantics preserved on nodes and relations. Our approach involves projecting the HINs into a low-dimensional vector space through two steps: node-level aggregation and type-level aggregation. The node-level aggregation employs an attention mechanism to create relation vectors by aggregating messages from neighborhoods connected via the same type of relation. The type-level aggregation leverages relation vectors to aggregate node representations. In particular, we introduce semantic-aware attention and relation-aware attention in the type-level aggregation to model the contributions of relation vectors, in order to simultaneously gain knowledge from node semantics and relational information. Unlike other approaches that rely on pre-defined meta-paths, our model can be readily applied to most real-world applications without requiring any domain knowledge. To validate the effectiveness of our proposed approach, we conducted extensive experiments on three public datasets. Experimental results demonstrate that the SR-HGN significantly outperforms state-of-the-art baselines on node classification and node clustering tasks.