Orionnnn/hub-stats huggingface dataset:
HoloPart: Generative 3D Part Amodal Segmentation
Authors: name: Yunhan Yang, name: Yuan-Chen Guo, name: Yukun Huang, name: Zi-Xin Zou, name: Zhipeng Yu, name: Yangguang Li, name: Yan-Pei Cao, name: Xihui Liu
Published: 2025-04-10 17:53:31

3D part amodal segmentation--decomposing a 3D shape into complete,
semantically meaningful parts, even when occluded--is a challenging but crucial
task for 3D content creation and understanding. Existing 3D part segmentation
methods only identify visible surface patches, limiting their utility. Inspired
by 2D amodal segmentation, we introduce this novel task to the 3D domain and
propose a practical, two-stage approach, addressing the key challenges of
inferring occluded 3D geometry, maintaining global shape consistency, and
handling diverse shapes with limited training data. First, we leverage existing
3D part segmentation to obtain initial, incomplete part segments. Second, we
introduce HoloPart, a novel diffusion-based model, to complete these segments
into full 3D parts. HoloPart utilizes a specialized architecture with local
attention to capture fine-grained part geometry and global shape context
attention to ensure overall shape consistency. We introduce new benchmarks
based on the ABO and PartObjaverse-Tiny datasets and demonstrate that HoloPart
significantly outperforms state-of-the-art shape completion methods. By
incorporating HoloPart with existing segmentation techniques, we achieve
promising results on 3D part amodal segmentation, opening new avenues for
applications in geometry editing, animation, and material assignment.
Fast Controlled Generation from Language Models with Adaptive Weighted
  Rejection Sampling
Authors: name: Benjamin Lipkin, name: Benjamin LeBrun, name: Jacob Hoover Vigly, name: João Loula, name: David R. MacIver, name: Li Du, name: Jason Eisner, name: Ryan Cotterell, name: Vikash Mansinghka, name: Timothy J. O'Donnell, name: Alexander K. Lew, name: Tim Vieira
Published: 2025-04-07 18:30:18

The dominant approach to generating from language models subject to some
constraint is locally constrained decoding (LCD), incrementally sampling tokens
at each time step such that the constraint is never violated. Typically, this
is achieved through token masking: looping over the vocabulary and excluding
non-conforming tokens. There are two important problems with this approach. (i)
Evaluating the constraint on every token can be prohibitively expensive -- LM
vocabularies often exceed 100,000 tokens. (ii) LCD can distort the global
distribution over strings, sampling tokens based only on local information,
even if they lead down dead-end paths. This work introduces a new algorithm
that addresses both these problems. First, to avoid evaluating a constraint on
the full vocabulary at each step of generation, we propose an adaptive
rejection sampling algorithm that typically requires orders of magnitude fewer
constraint evaluations. Second, we show how this algorithm can be extended to
produce low-variance, unbiased estimates of importance weights at a very small
additional cost -- estimates that can be soundly used within previously
proposed sequential Monte Carlo algorithms to correct for the myopic behavior
of local constraint enforcement. Through extensive empirical evaluation in
text-to-SQL, molecular synthesis, goal inference, pattern matching, and JSON
domains, we show that our approach is superior to state-of-the-art baselines,
supporting a broader class of constraints and improving both runtime and
performance. Additional theoretical and empirical analyses show that our
method's runtime efficiency is driven by its dynamic use of computation,
scaling with the divergence between the unconstrained and constrained LM, and
as a consequence, runtime improvements are greater for better models.
Caption Anything in Video: Fine-grained Object-centric Captioning via
  Spatiotemporal Multimodal Prompting
Authors: name: Yunlong Tang, name: Jing Bi, name: Chao Huang, name: Susan Liang, name: Daiki Shimada, name: Hang Hua, name: Yunzhong Xiao, name: Yizhi Song, name: Pinxin Liu, name: Mingqian Feng, name: Junjia Guo, name: Zhuo Liu, name: Luchuan Song, name: Ali Vosoughi, name: Jinxi He, name: Liu He, name: Zeliang Zhang, name: Jiebo Luo, name: Chenliang Xu
Published: 2025-04-07 22:35:36

We present CAT-V (Caption AnyThing in Video), a training-free framework for
fine-grained object-centric video captioning that enables detailed descriptions
of user-selected objects through time. CAT-V integrates three key components: a
Segmenter based on SAMURAI for precise object segmentation across frames, a
Temporal Analyzer powered by TRACE-Uni for accurate event boundary detection
and temporal analysis, and a Captioner using InternVL-2.5 for generating
detailed object-centric descriptions. Through spatiotemporal visual prompts and
chain-of-thought reasoning, our framework generates detailed, temporally-aware
descriptions of objects' attributes, actions, statuses, interactions, and
environmental contexts without requiring additional training data. CAT-V
supports flexible user interactions through various visual prompts (points,
bounding boxes, and irregular regions) and maintains temporal sensitivity by
tracking object states and interactions across different time segments. Our
approach addresses limitations of existing video captioning methods, which
either produce overly abstract descriptions or lack object-level precision,
enabling fine-grained, object-specific descriptions while maintaining temporal
coherence and spatial accuracy. The GitHub repository for this project is
available at https://github.com/yunlong10/CAT-V
Pretraining Language Models for Diachronic Linguistic Change Discovery
Authors: name: Elisabeth Fittschen, name: Sabrina Li, name: Tom Lippincott, name: Leshem Choshen, name: Craig Messner
Published: 2025-04-07 21:51:32

Large language models (LLMs) have shown potential as tools for scientific
discovery. This has engendered growing interest in their use in humanistic
disciplines, such as historical linguistics and literary studies. These fields
often construct arguments on the basis of delineations like genre, or more
inflexibly, time period. Although efforts have been made to restrict inference
to specific domains via fine-tuning or model editing, we posit that the only
true guarantee is domain-restricted pretraining -- typically, a data- and
compute-expensive proposition.
  We show that efficient pretraining techniques can produce useful models over
corpora too large for easy manual inspection but too small for "typical" LLM
approaches. We employ a novel date-attribution pipeline in order to obtain a
temporally-segmented dataset of five 10-million-word slices. We train two
corresponding five-model batteries over these corpus segments, efficient
pretraining and Llama3-8B parameter efficiently finetuned.
  We find that the pretrained models are faster to train than the finetuned
baselines and that they better respect the historical divisions of our corpus.
Emphasizing speed and precision over a-historical comprehensiveness enables a
number of novel approaches to hypothesis discovery and testing in our target
fields. Taking up diachronic linguistics as a testbed, we show that our method
enables the detection of a diverse set of phenomena, including en masse lexical
change, non-lexical (grammatical and morphological) change, and word sense
introduction/obsolescence. We provide a ready-to-use pipeline that allows
extension of our approach to other target fields with only minimal adaptation.
Masked Scene Modeling: Narrowing the Gap Between Supervised and
  Self-Supervised Learning in 3D Scene Understanding
Authors: name: Pedro Hermosilla, name: Christian Stippel, name: Leon Sick
Published: 2025-04-09 09:19:49

Self-supervised learning has transformed 2D computer vision by enabling
models trained on large, unannotated datasets to provide versatile
off-the-shelf features that perform similarly to models trained with labels.
However, in 3D scene understanding, self-supervised methods are typically only
used as a weight initialization step for task-specific fine-tuning, limiting
their utility for general-purpose feature extraction. This paper addresses this
shortcoming by proposing a robust evaluation protocol specifically designed to
assess the quality of self-supervised features for 3D scene understanding. Our
protocol uses multi-resolution feature sampling of hierarchical models to
create rich point-level representations that capture the semantic capabilities
of the model and, hence, are suitable for evaluation with linear probing and
nearest-neighbor methods. Furthermore, we introduce the first self-supervised
model that performs similarly to supervised models when only off-the-shelf
features are used in a linear probing setup. In particular, our model is
trained natively in 3D with a novel self-supervised approach based on a Masked
Scene Modeling objective, which reconstructs deep features of masked patches in
a bottom-up manner and is specifically tailored to hierarchical 3D models. Our
experiments not only demonstrate that our method achieves competitive
performance to supervised models, but also surpasses existing self-supervised
approaches by a large margin. The model and training code can be found at our
Github repository (https://github.com/phermosilla/msm).
A Unified Agentic Framework for Evaluating Conditional Image Generation
Authors: name: Jifang Wang, name: Xue Yang, name: Longyue Wang, name: Zhenran Xu, name: Yiyu Wang, name: Yaowei Wang, name: Weihua Luo, name: Kaifu Zhang, name: Baotian Hu, name: Min Zhang
Published: 2025-04-09 17:04:14

Conditional image generation has gained significant attention for its ability
to personalize content. However, the field faces challenges in developing
task-agnostic, reliable, and explainable evaluation metrics. This paper
introduces CIGEval, a unified agentic framework for comprehensive evaluation of
conditional image generation tasks. CIGEval utilizes large multimodal models
(LMMs) as its core, integrating a multi-functional toolbox and establishing a
fine-grained evaluation framework. Additionally, we synthesize evaluation
trajectories for fine-tuning, empowering smaller LMMs to autonomously select
appropriate tools and conduct nuanced analyses based on tool outputs.
Experiments across seven prominent conditional image generation tasks
demonstrate that CIGEval (GPT-4o version) achieves a high correlation of 0.4625
with human assessments, closely matching the inter-annotator correlation of
0.47. Moreover, when implemented with 7B open-source LMMs using only 2.3K
training trajectories, CIGEval surpasses the previous GPT-4o-based
state-of-the-art method. Case studies on GPT-4o image generation highlight
CIGEval's capability in identifying subtle issues related to subject
consistency and adherence to control guidance, indicating its great potential
for automating evaluation of image generation tasks with human-level
reliability.
WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environments
Authors: name: Jianhao Zheng, name: Zihan Zhu, name: Valentin Bieri, name: Marc Pollefeys, name: Songyou Peng, name: Iro Armeni
Published: 2025-04-04 19:19:40

We present WildGS-SLAM, a robust and efficient monocular RGB SLAM system
designed to handle dynamic environments by leveraging uncertainty-aware
geometric mapping. Unlike traditional SLAM systems, which assume static scenes,
our approach integrates depth and uncertainty information to enhance tracking,
mapping, and rendering performance in the presence of moving objects. We
introduce an uncertainty map, predicted by a shallow multi-layer perceptron and
DINOv2 features, to guide dynamic object removal during both tracking and
mapping. This uncertainty map enhances dense bundle adjustment and Gaussian map
optimization, improving reconstruction accuracy. Our system is evaluated on
multiple datasets and demonstrates artifact-free view synthesis. Results
showcase WildGS-SLAM's superior performance in dynamic environments compared to
state-of-the-art methods.
RuOpinionNE-2024: Extraction of Opinion Tuples from Russian News Texts
Authors: name: Natalia Loukachevitch, name: Natalia Tkachenko, name: Anna Lapanitsyna, name: Mikhail Tikhomirov, name: Nicolay Rusnachenko
Published: 2025-04-09 14:54:00

In this paper, we introduce the Dialogue Evaluation shared task on extraction
of structured opinions from Russian news texts. The task of the contest is to
extract opinion tuples for a given sentence; the tuples are composed of a
sentiment holder, its target, an expression and sentiment from the holder to
the target. In total, the task received more than 100 submissions. The
participants experimented mainly with large language models in zero-shot,
few-shot and fine-tuning formats. The best result on the test set was obtained
with fine-tuning of a large language model. We also compared 30 prompts and 11
open source language models with 3-32 billion parameters in the 1-shot and
10-shot settings and found the best models and prompts.
VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement
  Fine-Tuning
Authors: name: Xinhao Li, name: Ziang Yan, name: Desen Meng, name: Lu Dong, name: Xiangyu Zeng, name: Yinan He, name: Yali Wang, name: Yu Qiao, name: Yi Wang, name: Limin Wang
Published: 2025-04-09 15:09:27

Recent advancements in reinforcement learning have significantly advanced the
reasoning capabilities of multimodal large language models (MLLMs). While
approaches such as Group Relative Policy Optimization (GRPO) and rule-based
reward mechanisms demonstrate promise in text and image domains, their
application to video understanding remains limited. This paper presents a
systematic exploration of Reinforcement Fine-Tuning (RFT) with GRPO for video
MLLMs, aiming to enhance spatio-temporal perception while maintaining general
capabilities. Our experiments reveal that RFT is highly data-efficient for
task-specific improvements. Through multi-task RFT on spatio-temporal
perception objectives with limited samples, we develop VideoChat-R1, a powerful
video MLLM that achieves state-of-the-art performance on spatio-temporal
perception tasks without sacrificing chat ability, while exhibiting emerging
spatio-temporal reasoning abilities. Compared to Qwen2.5-VL-7B, VideoChat-R1
boosts performance several-fold in tasks like temporal grounding (+31.8) and
object tracking (+31.2). Additionally, it significantly improves on general QA
benchmarks such as VideoMME (+0.9), MVBench (+1.0), and Perception Test (+0.9).
Our findings underscore the potential of RFT for specialized task enhancement
of Video MLLMs. We hope our work offers valuable insights for future RL
research in video MLLMs.
Are We Done with Object-Centric Learning?
Authors: name: Alexander Rubinstein, name: Ameya Prabhu, name: Matthias Bethge, name: Seong Joon Oh
Published: 2025-04-09 17:59:05

Object-centric learning (OCL) seeks to learn representations that only encode
an object, isolated from other objects or background cues in a scene. This
approach underpins various aims, including out-of-distribution (OOD)
generalization, sample-efficient composition, and modeling of structured
environments. Most research has focused on developing unsupervised mechanisms
that separate objects into discrete slots in the representation space,
evaluated using unsupervised object discovery. However, with recent
sample-efficient segmentation models, we can separate objects in the pixel
space and encode them independently. This achieves remarkable zero-shot
performance on OOD object discovery benchmarks, is scalable to foundation
models, and can handle a variable number of slots out-of-the-box. Hence, the
goal of OCL methods to obtain object-centric representations has been largely
achieved. Despite this progress, a key question remains: How does the ability
to separate objects within a scene contribute to broader OCL objectives, such
as OOD generalization? We address this by investigating the OOD generalization
challenge caused by spurious background cues through the lens of OCL. We
propose a novel, training-free probe called Object-Centric
Classification with Applied Masks (OCCAM), demonstrating that
segmentation-based encoding of individual objects significantly outperforms
slot-based OCL methods. However, challenges in real-world applications remain.
We provide the toolbox for the OCL community to use scalable object-centric
representations, and focus on practical applications and fundamental questions,
such as understanding object perception in human cognition. Our code is
available https://github.com/AlexanderRubinstein/OCCAM{here}.
Self-Steering Language Models
Authors: name: Gabriel Grand, name: Joshua B. Tenenbaum, name: Vikash K. Mansinghka, name: Alexander K. Lew, name: Jacob Andreas
Published: 2025-04-09 17:54:22

While test-time reasoning enables language models to tackle complex tasks,
searching or planning in natural language can be slow, costly, and error-prone.
But even when LMs struggle to emulate the precise reasoning steps needed to
solve a problem, they often excel at describing its abstract structure--both
how to verify solutions and how to search for them. This paper introduces
DisCIPL, a method for "self-steering" LMs where a Planner model generates a
task-specific inference program that is executed by a population of Follower
models. Our approach equips LMs with the ability to write recursive search
procedures that guide LM inference, enabling new forms of verifiable and
efficient reasoning. When instantiated with a small Follower (e.g.,
Llama-3.2-1B), DisCIPL matches (and sometimes outperforms) much larger models,
including GPT-4o and o1, on challenging constrained generation tasks. In
decoupling planning from execution, our work opens up a design space of
highly-parallelized Monte Carlo inference strategies that outperform standard
best-of-N sampling, require no finetuning, and can be implemented automatically
by existing LMs.
RobustDexGrasp: Robust Dexterous Grasping of General Objects from
  Single-view Perception
Authors: name: Hui Zhang, name: Zijian Wu, name: Linyi Huang, name: Sammy Christen, name: Jie Song
Published: 2025-04-07 17:38:19

Robust grasping of various objects from single-view perception is fundamental
for dexterous robots. Previous works often rely on fully observable objects,
expert demonstrations, or static grasping poses, which restrict their
generalization ability and adaptability to external disturbances. In this
paper, we present a reinforcement-learning-based framework that enables
zero-shot dynamic dexterous grasping of a wide range of unseen objects from
single-view perception, while performing adaptive motions to external
disturbances. We utilize a hand-centric object representation for shape feature
extraction that emphasizes interaction-relevant local shapes, enhancing
robustness to shape variance and uncertainty. To enable effective hand
adaptation to disturbances with limited observations, we propose a mixed
curriculum learning strategy, which first utilizes imitation learning to
distill a policy trained with privileged real-time visual-tactile feedback, and
gradually transfers to reinforcement learning to learn adaptive motions under
disturbances caused by observation noises and dynamic randomization. Our
experiments demonstrate strong generalization in grasping unseen objects with
random poses, achieving success rates of 97.0% across 247,786 simulated objects
and 94.6% across 512 real objects. We also demonstrate the robustness of our
method to various disturbances, including unobserved object movement and
external forces, through both quantitative and qualitative evaluations. Project
Page: https://zdchan.github.io/Robust_DexGrasp/
OmniCaptioner: One Captioner to Rule Them All
Authors: name: Yiting Lu, name: Jiakang Yuan, name: Zhen Li, name: Shitian Zhao, name: Qi Qin, name: Xinyue Li, name: Le Zhuo, name: Licheng Wen, name: Dongyang Liu, name: Yuewen Cao, name: Xiangchao Yan, name: Xin Li, name: Botian Shi, name: Tao Chen, name: Zhibo Chen, name: Lei Bai, name: Bo Zhang, name: Peng Gao
Published: 2025-04-09 17:58:58

We propose OmniCaptioner, a versatile visual captioning framework for
generating fine-grained textual descriptions across a wide variety of visual
domains. Unlike prior methods limited to specific image types (e.g., natural
images or geometric visuals), our framework provides a unified solution for
captioning natural images, visual text (e.g., posters, UIs, textbooks), and
structured visuals (e.g., documents, tables, charts). By converting low-level
pixel information into semantically rich textual representations, our framework
bridges the gap between visual and textual modalities. Our results highlight
three key advantages: (i) Enhanced Visual Reasoning with LLMs, where
long-context captions of visual modalities empower LLMs, particularly the
DeepSeek-R1 series, to reason effectively in multimodal scenarios; (ii)
Improved Image Generation, where detailed captions improve tasks like
text-to-image generation and image transformation; and (iii) Efficient
Supervised Fine-Tuning (SFT), which enables faster convergence with less data.
We believe the versatility and adaptability of OmniCaptioner can offer a new
perspective for bridging the gap between language and visual modalities.
DiTaiListener: Controllable High Fidelity Listener Video Generation with
  Diffusion
Authors: name: Maksim Siniukov, name: Di Chang, name: Minh Tran, name: Hongkun Gong, name: Ashutosh Chaubey, name: Mohammad Soleymani
Published: 2025-04-05 01:19:46

Generating naturalistic and nuanced listener motions for extended
interactions remains an open problem. Existing methods often rely on
low-dimensional motion codes for facial behavior generation followed by
photorealistic rendering, limiting both visual fidelity and expressive
richness. To address these challenges, we introduce DiTaiListener, powered by a
video diffusion model with multimodal conditions. Our approach first generates
short segments of listener responses conditioned on the speaker's speech and
facial motions with DiTaiListener-Gen. It then refines the transitional frames
via DiTaiListener-Edit for a seamless transition. Specifically,
DiTaiListener-Gen adapts a Diffusion Transformer (DiT) for the task of listener
head portrait generation by introducing a Causal Temporal Multimodal Adapter
(CTM-Adapter) to process speakers' auditory and visual cues. CTM-Adapter
integrates speakers' input in a causal manner into the video generation process
to ensure temporally coherent listener responses. For long-form video
generation, we introduce DiTaiListener-Edit, a transition refinement
video-to-video diffusion model. The model fuses video segments into smooth and
continuous videos, ensuring temporal consistency in facial expressions and
image quality when merging short video segments produced by DiTaiListener-Gen.
Quantitatively, DiTaiListener achieves the state-of-the-art performance on
benchmark datasets in both photorealism (+73.8% in FID on RealTalk) and motion
representation (+6.1% in FD metric on VICO) spaces. User studies confirm the
superior performance of DiTaiListener, with the model being the clear
preference in terms of feedback, diversity, and smoothness, outperforming
competitors by a significant margin.