Orionnnn/hub-stats huggingface dataset:
Decompositional Neural Scene Reconstruction with Generative Diffusion
  Prior
Authors: name: Junfeng Ni, name: Yu Liu, name: Ruijie Lu, name: Zirui Zhou, name: Song-Chun Zhu, name: Yixin Chen, name: Siyuan Huang
Published: 2025-03-19 02:11:31

Decompositional reconstruction of 3D scenes, with complete shapes and
detailed texture of all objects within, is intriguing for downstream
applications but remains challenging, particularly with sparse views as input.
Recent approaches incorporate semantic or geometric regularization to address
this issue, but they suffer significant degradation in underconstrained areas
and fail to recover occluded regions. We argue that the key to solving this
problem lies in supplementing missing information for these areas. To this end,
we propose DP-Recon, which employs diffusion priors in the form of Score
Distillation Sampling (SDS) to optimize the neural representation of each
individual object under novel views. This provides additional information for
the underconstrained areas, but directly incorporating diffusion prior raises
potential conflicts between the reconstruction and generative guidance.
Therefore, we further introduce a visibility-guided approach to dynamically
adjust the per-pixel SDS loss weights. Together these components enhance both
geometry and appearance recovery while remaining faithful to input images.
Extensive experiments across Replica and ScanNet++ demonstrate that our method
significantly outperforms SOTA methods. Notably, it achieves better object
reconstruction under 10 views than the baselines under 100 views. Our method
enables seamless text-based editing for geometry and appearance through SDS
optimization and produces decomposed object meshes with detailed UV maps that
support photorealistic Visual effects (VFX) editing. The project page is
available at https://dp-recon.github.io/.
VERIFY: A Benchmark of Visual Explanation and Reasoning for
  Investigating Multimodal Reasoning Fidelity
Authors: name: Jing Bi, name: Junjia Guo, name: Susan Liang, name: Guangyu Sun, name: Luchuan Song, name: Yunlong Tang, name: Jinxi He, name: Jiarui Wu, name: Ali Vosoughi, name: Chen Chen, name: Chenliang Xu
Published: 2025-03-14 16:26:11

Visual reasoning is central to human cognition, enabling individuals to
interpret and abstractly understand their environment. Although recent
Multimodal Large Language Models (MLLMs) have demonstrated impressive
performance across language and vision-language tasks, existing benchmarks
primarily measure recognition-based skills and inadequately assess true visual
reasoning capabilities. To bridge this critical gap, we introduce VERIFY, a
benchmark explicitly designed to isolate and rigorously evaluate the visual
reasoning capabilities of state-of-the-art MLLMs. VERIFY compels models to
reason primarily from visual information, providing minimal textual context to
reduce reliance on domain-specific knowledge and linguistic biases. Each
problem is accompanied by a human-annotated reasoning path, making it the first
to provide in-depth evaluation of model decision-making processes.
Additionally, we propose novel metrics that assess visual reasoning fidelity
beyond mere accuracy, highlighting critical imbalances in current model
reasoning patterns. Our comprehensive benchmarking of leading MLLMs uncovers
significant limitations, underscoring the need for a balanced and holistic
approach to both perception and reasoning. For more teaser and testing, visit
our project page (https://verify-eqh.pages.dev/).
Optimizing Decomposition for Optimal Claim Verification
Authors: name: Yining Lu, name: Noah Ziems, name: Hy Dang, name: Meng Jiang
Published: 2025-03-19 15:56:21

Current research on the Decompose-Then-Verify paradigm for
evaluating the factuality of long-form text typically treats decomposition and
verification in isolation, overlooking their interactions and potential
misalignment. We find that existing decomposition policies, typically
hand-crafted demonstrations, do not align well with downstream verifiers in
terms of atomicity -- a novel metric quantifying information density -- leading
to suboptimal verification results. We formulate finding the optimal
decomposition policy for optimal verification as a bilevel optimization
problem. To approximate a solution for this strongly NP-hard problem, we
propose dynamic decomposition, a reinforcement learning framework that
leverages verifier feedback to learn a policy for dynamically decomposing
claims to verifier-preferred atomicity. Experimental results show that dynamic
decomposition outperforms existing decomposition policies, improving
verification confidence by 0.07 and accuracy by 0.12 (on a 0-1 scale) on
average across varying verifiers, datasets, and atomcities of input claims.
MetaLadder: Ascending Mathematical Solution Quality via
  Analogical-Problem Reasoning Transfer
Authors: name: Honglin Lin, name: Zhuoshi Pan, name: Yu Li, name: Qizhi Pei, name: Xin Gao, name: Mengzhang Cai, name: Conghui He, name: Lijun Wu
Published: 2025-03-19 04:36:35

Large Language Models (LLMs) have demonstrated promising capabilities in
solving mathematical reasoning tasks, leveraging Chain-of-Thought (CoT) data as
a vital component in guiding answer generation. Current paradigms typically
generate CoT and answers directly for a given problem, diverging from human
problem-solving strategies to some extent. Humans often solve problems by
recalling analogous cases and leveraging their solutions to reason about the
current task. Inspired by this cognitive process, we propose
MetaLadder, a novel framework that explicitly prompts LLMs to recall
and reflect on meta-problems, those structurally or semantically analogous
problems, alongside their CoT solutions before addressing the target problem.
Additionally, we introduce a problem-restating mechanism to enhance the model's
comprehension of the target problem by regenerating the original question,
which further improves reasoning accuracy. Therefore, the model can achieve
reasoning transfer from analogical problems, mimicking human-like "learning
from examples" and generalization abilities. Extensive experiments on
mathematical benchmarks demonstrate that our MetaLadder significantly boosts
LLMs' problem-solving accuracy, largely outperforming standard CoT-based
methods (10.3\% accuracy gain) and other methods. Our code and data
has been released at https://github.com/LHL3341/MetaLadder.
CURIE: Evaluating LLMs On Multitask Scientific Long Context
  Understanding and Reasoning
Authors: name: Hao Cui, name: Zahra Shamsi, name: Gowoon Cheon, name: Xuejian Ma, name: Shutong Li, name: Maria Tikhanovskaya, name: Peter Norgaard, name: Nayantara Mudur, name: Martyna Plomecka, name: Paul Raccuglia, name: Yasaman Bahri, name: Victor V. Albert, name: Pranesh Srinivasan, name: Haining Pan, name: Philippe Faist, name: Brian Rohr, name: Michael J. Statt, name: Dan Morris, name: Drew Purves, name: Elise Kleeman, name: Ruth Alcantara, name: Matthew Abraham, name: Muqthar Mohammad, name: Ean Phing VanLee, name: Chenfei Jiang, name: Elizabeth Dorfman, name: Eun-Ah Kim, name: Michael P Brenner, name: Viren Jain, name: Sameera Ponda, name: Subhashini Venugopalan
Published: 2025-03-14 17:53:03

Scientific problem-solving involves synthesizing information while applying
expert knowledge. We introduce CURIE, a scientific long-Context
Understanding,Reasoning and Information Extraction benchmark to measure the
potential of Large Language Models (LLMs) in scientific problem-solving and
assisting scientists in realistic workflows. This benchmark introduces ten
challenging tasks with a total of 580 problems and solution pairs curated by
experts in six disciplines - materials science, condensed matter physics,
quantum computing, geospatial analysis, biodiversity, and proteins - covering
both experimental and theoretical work-flows in science. We evaluate a range of
closed and open LLMs on tasks in CURIE which requires domain expertise,
comprehension of long in-context information,and multi-step reasoning. While
Gemini Flash 2.0 and Claude-3 show consistent high comprehension across
domains, the popular GPT-4o and command-R+ fail dramatically on protein
sequencing tasks. With the best performance at 32% there is much room for
improvement for all models. We hope that insights gained from CURIE can guide
the future development of LLMs in sciences. Evaluation code and data are in
https://github.com/google/curie
Unlock Pose Diversity: Accurate and Efficient Implicit Keypoint-based
  Spatiotemporal Diffusion for Audio-driven Talking Portrait
Authors: name: Chaolong Yang, name: Kai Yao, name: Yuyao Yan, name: Chenru Jiang, name: Weiguang Zhao, name: Jie Sun, name: Guangliang Cheng, name: Yifei Zhang, name: Bin Dong, name: Kaizhu Huang
Published: 2025-03-17 09:18:31

Audio-driven single-image talking portrait generation plays a crucial role in
virtual reality, digital human creation, and filmmaking. Existing approaches
are generally categorized into keypoint-based and image-based methods.
Keypoint-based methods effectively preserve character identity but struggle to
capture fine facial details due to the fixed points limitation of the 3D
Morphable Model. Moreover, traditional generative networks face challenges in
establishing causality between audio and keypoints on limited datasets,
resulting in low pose diversity. In contrast, image-based approaches produce
high-quality portraits with diverse details using the diffusion network but
incur identity distortion and expensive computational costs. In this work, we
propose KDTalker, the first framework to combine unsupervised implicit 3D
keypoint with a spatiotemporal diffusion model. Leveraging unsupervised
implicit 3D keypoints, KDTalker adapts facial information densities, allowing
the diffusion process to model diverse head poses and capture fine facial
details flexibly. The custom-designed spatiotemporal attention mechanism
ensures accurate lip synchronization, producing temporally consistent,
high-quality animations while enhancing computational efficiency. Experimental
results demonstrate that KDTalker achieves state-of-the-art performance
regarding lip synchronization accuracy, head pose diversity, and execution
efficiency.Our codes are available at https://github.com/chaolongy/KDTalker.
GKG-LLM: A Unified Framework for Generalized Knowledge Graph
  Construction
Authors: name: Jian Zhang, name: Bifan Wei, name: Shihao Qi, name: haiping Zhu, name: Jun Liu, name: Qika Lin
Published: 2025-03-14 09:23:22

The construction of Generalized Knowledge Graph (GKG), including knowledge
graph, event knowledge graph and commonsense knowledge graph, is fundamental
for various natural language processing tasks. Current studies typically
construct these types of graph separately, overlooking holistic insights and
potential unification that could be beneficial in computing resources and usage
perspectives. However, a key challenge in developing a unified framework for
GKG is obstacles arising from task-specific differences. In this study, we
propose a unified framework for constructing generalized knowledge graphs to
address this challenge. First, we collect data from 15 sub-tasks in 29 datasets
across the three types of graphs, categorizing them into in-sample,
counter-task, and out-of-distribution (OOD) data. Then, we propose a
three-stage curriculum learning fine-tuning framework, by iteratively injecting
knowledge from the three types of graphs into the Large Language Models.
Extensive experiments show that our proposed model improves the construction of
all three graph types across in-domain, OOD and counter-task data.
φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time
  Exploration and Exploitation
Authors: name: Fangzhi Xu, name: Hang Yan, name: Chang Ma, name: Haiteng Zhao, name: Jun Liu, name: Qika Lin, name: Zhiyong Wu
Published: 2025-03-17 15:38:33

Inference-time optimization scales computation to derive deliberate reasoning
steps for effective performance. While previous search-based strategies address
the short-sightedness of auto-regressive generation, the vast search space
leads to excessive exploration and insufficient exploitation. To strike an
efficient balance to derive the optimal step, we frame the decoding strategy as
foresight sampling, leveraging simulated future steps to obtain globally
optimal step estimation. Built on it, we propose a novel decoding strategy,
named phi-Decoding. To provide a precise and expressive estimation of step
value, phi-Decoding approximates two distributions via foresight and
clustering. Sampling from the joint distribution, the optimal steps can be
selected for exploitation. To support adaptive computation allocation, we
propose in-width and in-depth pruning strategies, featuring a light-weight
solution to achieve inference efficiency. Extensive experiments across seven
benchmarks show phi-Decoding outperforms strong baselines in both
performance and efficiency. Additional analysis demonstrates its generalization
across various LLMs and scalability across a wide range of computing budgets.
The code will be released at https://github.com/xufangzhi/phi-Decoding, and the
open-source PyPI package is coming soon.
LEGION: Learning to Ground and Explain for Synthetic Image Detection
Authors: name: Hengrui Kang, name: Siwei Wen, name: Zichen Wen, name: Junyan Ye, name: Weijia Li, name: Peilin Feng, name: Baichuan Zhou, name: Bin Wang, name: Dahua Lin, name: Linfeng Zhang, name: Conghui He
Published: 2025-03-19 14:37:21

The rapid advancements in generative technology have emerged as a
double-edged sword. While offering powerful tools that enhance convenience,
they also pose significant social concerns. As defenders, current synthetic
image detection methods often lack artifact-level textual interpretability and
are overly focused on image manipulation detection, and current datasets
usually suffer from outdated generators and a lack of fine-grained annotations.
In this paper, we introduce SynthScars, a high-quality and diverse dataset
consisting of 12,236 fully synthetic images with human-expert annotations. It
features 4 distinct image content types, 3 categories of artifacts, and
fine-grained annotations covering pixel-level segmentation, detailed textual
explanations, and artifact category labels. Furthermore, we propose LEGION
(LEarning to Ground and explain for Synthetic Image detectiON), a multimodal
large language model (MLLM)-based image forgery analysis framework that
integrates artifact detection, segmentation, and explanation. Building upon
this capability, we further explore LEGION as a controller, integrating it into
image refinement pipelines to guide the generation of higher-quality and more
realistic images. Extensive experiments show that LEGION outperforms existing
methods across multiple benchmarks, particularly surpassing the second-best
traditional expert on SynthScars by 3.31% in mIoU and 7.75% in F1 score.
Moreover, the refined images generated under its guidance exhibit stronger
alignment with human preferences. The code, model, and dataset will be
released.
Mitigating Visual Forgetting via Take-along Visual Conditioning for
  Multi-modal Long CoT Reasoning
Authors: name: Hai-Long Sun, name: Zhun Sun, name: Houwen Peng, name: Han-Jia Ye
Published: 2025-03-17 16:45:12

Recent advancements in Large Language Models (LLMs) have demonstrated
enhanced reasoning capabilities, evolving from Chain-of-Thought (CoT) prompting
to advanced, product-oriented solutions like OpenAI o1. During our
re-implementation of this model, we noticed that in multimodal tasks requiring
visual input (e.g., geometry problems), Multimodal LLMs (MLLMs) struggle to
maintain focus on the visual information, in other words, MLLMs suffer from a
gradual decline in attention to visual information as reasoning progresses,
causing text-over-relied outputs. To investigate this, we ablate image inputs
during long-chain reasoning. Concretely, we truncate the reasoning process
midway, then re-complete the reasoning process with the input image removed. We
observe only a ~2% accuracy drop on MathVista's test-hard subset, revealing the
model's textual outputs dominate the following reasoning process. Motivated by
this, we propose Take-along Visual Conditioning (TVC), a strategy that shifts
image input to critical reasoning stages and compresses redundant visual tokens
via dynamic pruning. This methodology helps the model retain attention to the
visual components throughout the reasoning. Our approach achieves
state-of-the-art performance on average across five mathematical reasoning
benchmarks (+3.4% vs previous sota), demonstrating the effectiveness of TVC in
enhancing multimodal reasoning systems.
Efficient Personalization of Quantized Diffusion Model without
  Backpropagation
Authors: name: Hoigi Seo, name: Wongi Jeong, name: Kyungryeol Lee, name: Se Young Chun
Published: 2025-03-19 03:45:37

Diffusion models have shown remarkable performance in image synthesis, but
they demand extensive computational and memory resources for training,
fine-tuning and inference. Although advanced quantization techniques have
successfully minimized memory usage for inference, training and fine-tuning
these quantized models still require large memory possibly due to
dequantization for accurate computation of gradients and/or backpropagation for
gradient-based algorithms. However, memory-efficient fine-tuning is
particularly desirable for applications such as personalization that often must
be run on edge devices like mobile phones with private data. In this work, we
address this challenge by quantizing a diffusion model with personalization via
Textual Inversion and by leveraging a zeroth-order optimization on
personalization tokens without dequantization so that it does not require
gradient and activation storage for backpropagation that consumes considerable
memory. Since a gradient estimation using zeroth-order optimization is quite
noisy for a single or a few images in personalization, we propose to denoise
the estimated gradient by projecting it onto a subspace that is constructed
with the past history of the tokens, dubbed Subspace Gradient. In addition, we
investigated the influence of text embedding in image generation, leading to
our proposed time steps sampling, dubbed Partial Uniform Timestep Sampling for
sampling with effective diffusion timesteps. Our method achieves comparable
performance to prior methods in image and text alignment scores for
personalizing Stable Diffusion with only forward passes while reducing training
memory demand up to 8.2times.
DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement
  Learning
Authors: name: Ruowen Zhao, name: Junliang Ye, name: Zhengyi Wang, name: Guangce Liu, name: Yiwen Chen, name: Yikai Wang, name: Jun Zhu
Published: 2025-03-19 14:39:30

Triangle meshes play a crucial role in 3D applications for efficient
manipulation and rendering. While auto-regressive methods generate structured
meshes by predicting discrete vertex tokens, they are often constrained by
limited face counts and mesh incompleteness. To address these challenges, we
propose DeepMesh, a framework that optimizes mesh generation through two key
innovations: (1) an efficient pre-training strategy incorporating a novel
tokenization algorithm, along with improvements in data curation and
processing, and (2) the introduction of Reinforcement Learning (RL) into 3D
mesh generation to achieve human preference alignment via Direct Preference
Optimization (DPO). We design a scoring standard that combines human evaluation
with 3D metrics to collect preference pairs for DPO, ensuring both visual
appeal and geometric accuracy. Conditioned on point clouds and images, DeepMesh
generates meshes with intricate details and precise topology, outperforming
state-of-the-art methods in both precision and quality. Project page:
https://zhaorw02.github.io/DeepMesh/
ELTEX: A Framework for Domain-Driven Synthetic Data Generation
Authors: name: Arina Razmyslovich, name: Kseniia Murasheva, name: Sofia Sedlova, name: Julien Capitaine, name: Eugene Dmitriev
Published: 2025-03-19 09:46:54

We present ELTEX (Efficient LLM Token Extraction), a domain-driven framework
for generating high-quality synthetic training data in specialized domains.
While Large Language Models (LLMs) have shown impressive general capabilities,
their performance in specialized domains like cybersecurity remains limited by
the scarcity of domain-specific training data. ELTEX addresses this challenge
by systematically integrating explicit domain indicator extraction with dynamic
prompting to preserve critical domain knowledge throughout the generation
process. We demonstrate ELTEX's effectiveness in the context of
blockchain-related cyberattack detection, where we fine-tune Gemma-2B using
various combinations of real and ELTEX-generated data. Our results show that
the ELTEX-enhanced model achieves performance competitive with GPT-4 across
both standard classification metrics and uncertainty calibration, while
requiring significantly fewer computational resources. We release a curated
synthetic dataset of social media texts for cyberattack detection in
blockchain. Our work demonstrates that domain-driven synthetic data generation
can effectively bridge the performance gap between resource-efficient models
and larger architectures in specialized domains.
MusicInfuser: Making Video Diffusion Listen and Dance
Authors: name: Susung Hong, name: Ira Kemelmacher-Shlizerman, name: Brian Curless, name: Steven M. Seitz
Published: 2025-03-18 17:59:58

We introduce MusicInfuser, an approach for generating high-quality dance
videos that are synchronized to a specified music track. Rather than attempting
to design and train a new multimodal audio-video model, we show how existing
video diffusion models can be adapted to align with musical inputs by
introducing lightweight music-video cross-attention and a low-rank adapter.
Unlike prior work requiring motion capture data, our approach fine-tunes only
on dance videos. MusicInfuser achieves high-quality music-driven video
generation while preserving the flexibility and generative capabilities of the
underlying models. We introduce an evaluation framework using Video-LLMs to
assess multiple dimensions of dance generation quality. The project page and
code are available at https://susunghong.github.io/MusicInfuser.
Temporal Regularization Makes Your Video Generator Stronger
Authors: name: Harold Haodong Chen, name: Haojian Huang, name: Xianfeng Wu, name: Yexin Liu, name: Yajing Bai, name: Wen-Jie Shu, name: Harry Yang, name: Ser-Nam Lim
Published: 2025-03-19 16:59:32

Temporal quality is a critical aspect of video generation, as it ensures
consistent motion and realistic dynamics across frames. However, achieving high
temporal coherence and diversity remains challenging. In this work, we explore
temporal augmentation in video generation for the first time, and introduce
FluxFlow for initial investigation, a strategy designed to enhance temporal
quality. Operating at the data level, FluxFlow applies controlled temporal
perturbations without requiring architectural modifications. Extensive
experiments on UCF-101 and VBench benchmarks demonstrate that FluxFlow
significantly improves temporal coherence and diversity across various video
generation models, including U-Net, DiT, and AR-based architectures, while
preserving spatial fidelity. These findings highlight the potential of temporal
augmentation as a simple yet effective approach to advancing video generation
quality.