Orionnnn/hub-stats huggingface dataset:
FRESA:Feedforward Reconstruction of Personalized Skinned Avatars from
  Few Images
Authors: name: Rong Wang, name: Fabian Prada, name: Ziyan Wang, name: Zhongshi Jiang, name: Chengxiang Yin, name: Junxuan Li, name: Shunsuke Saito, name: Igor Santesteban, name: Javier Romero, name: Rohan Joshi, name: Hongdong Li, name: Jason Saragih, name: Yaser Sheikh
Published: 2025-03-24 23:20:47

We present a novel method for reconstructing personalized 3D human avatars
with realistic animation from only a few images. Due to the large variations in
body shapes, poses, and cloth types, existing methods mostly require hours of
per-subject optimization during inference, which limits their practical
applications. In contrast, we learn a universal prior from over a thousand
clothed humans to achieve instant feedforward generation and zero-shot
generalization. Specifically, instead of rigging the avatar with shared
skinning weights, we jointly infer personalized avatar shape, skinning weights,
and pose-dependent deformations, which effectively improves overall geometric
fidelity and reduces deformation artifacts. Moreover, to normalize pose
variations and resolve coupled ambiguity between canonical shapes and skinning
weights, we design a 3D canonicalization process to produce pixel-aligned
initial conditions, which helps to reconstruct fine-grained geometric details.
We then propose a multi-frame feature aggregation to robustly reduce artifacts
introduced in canonicalization and fuse a plausible avatar preserving
person-specific identities. Finally, we train the model in an end-to-end
framework on a large-scale capture dataset, which contains diverse human
subjects paired with high-quality 3D scans. Extensive experiments show that our
method generates more authentic reconstruction and animation than
state-of-the-arts, and can be directly generalized to inputs from casually
taken phone photos. Project page and code is available at
https://github.com/rongakowang/FRESA.
LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary
  Semantic Segmentation
Authors: name: Vladan Stojnić, name: Yannis Kalantidis, name: Jiří Matas, name: Giorgos Tolias
Published: 2025-03-25 15:47:13

We propose a training-free method for open-vocabulary semantic segmentation
using Vision-and-Language Models (VLMs). Our approach enhances the initial
per-patch predictions of VLMs through label propagation, which jointly
optimizes predictions by incorporating patch-to-patch relationships. Since VLMs
are primarily optimized for cross-modal alignment and not for intra-modal
similarity, we use a Vision Model (VM) that is observed to better capture these
relationships. We address resolution limitations inherent to patch-based
encoders by applying label propagation at the pixel level as a refinement step,
significantly improving segmentation accuracy near class boundaries. Our
method, called LPOSS+, performs inference over the entire image, avoiding
window-based processing and thereby capturing contextual interactions across
the full image. LPOSS+ achieves state-of-the-art performance among
training-free methods, across a diverse set of datasets. Code:
https://github.com/vladan-stojnic/LPOSS
Towards a Unified Copernicus Foundation Model for Earth Vision
Authors: name: Yi Wang, name: Zhitong Xiong, name: Chenying Liu, name: Adam J. Stewart, name: Thomas Dujardin, name: Nikolaos Ioannis Bountos, name: Angelos Zavras, name: Franziska Gerken, name: Ioannis Papoutsis, name: Laura Leal-Taixé, name: Xiao Xiang Zhu
Published: 2025-03-14 20:16:48

Advances in Earth observation (EO) foundation models have unlocked the
potential of big satellite data to learn generic representations from space,
benefiting a wide range of downstream applications crucial to our planet.
However, most existing efforts remain limited to fixed spectral sensors, focus
solely on the Earth's surface, and overlook valuable metadata beyond imagery.
In this work, we take a step towards next-generation EO foundation models with
three key components: 1) Copernicus-Pretrain, a massive-scale pretraining
dataset that integrates 18.7M aligned images from all major Copernicus Sentinel
missions, spanning from the Earth's surface to its atmosphere; 2)
Copernicus-FM, a unified foundation model capable of processing any spectral or
non-spectral sensor modality using extended dynamic hypernetworks and flexible
metadata encoding; and 3) Copernicus-Bench, a systematic evaluation benchmark
with 15 hierarchical downstream tasks ranging from preprocessing to specialized
applications for each Sentinel mission. Our dataset, model, and benchmark
greatly improve the scalability, versatility, and multimodal adaptability of EO
foundation models, while also creating new opportunities to connect EO,
weather, and climate research. Codes, datasets and models are available at
https://github.com/zhu-xlab/Copernicus-FM.
Strong Baseline: Multi-UAV Tracking via YOLOv12 with BoT-SORT-ReID
Authors: name: Yu-Hsi Chen
Published: 2025-03-21 15:40:18

Detecting and tracking multiple unmanned aerial vehicles (UAVs) in thermal
infrared video is inherently challenging due to low contrast, environmental
noise, and small target sizes. This paper provides a straightforward approach
to address multi-UAV tracking in thermal infrared video, leveraging recent
advances in detection and tracking. Instead of relying on the YOLOv5 with the
DeepSORT pipeline, we present a tracking framework built on YOLOv12 and
BoT-SORT, enhanced with tailored training and inference strategies. We evaluate
our approach following the metrics from the 4th Anti-UAV Challenge and
demonstrate competitive performance. Notably, we achieve strong results without
using contrast enhancement or temporal information fusion to enrich UAV
features, highlighting our approach as a "Strong Baseline" for the multi-UAV
tracking task. We provide implementation details, in-depth experimental
analysis, and a discussion of potential improvements. The code is available at
https://github.com/wish44165/YOLOv12-BoT-SORT-ReID .
Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection
  with Artifact Explanation
Authors: name: Siwei Wen, name: Junyan Ye, name: Peilin Feng, name: Hengrui Kang, name: Zichen Wen, name: Yize Chen, name: Jiang Wu, name: Wenjun Wu, name: Conghui He, name: Weijia Li
Published: 2025-03-19 05:14:44

With the rapid advancement of Artificial Intelligence Generated Content
(AIGC) technologies, synthetic images have become increasingly prevalent in
everyday life, posing new challenges for authenticity assessment and detection.
Despite the effectiveness of existing methods in evaluating image authenticity
and locating forgeries, these approaches often lack human interpretability and
do not fully address the growing complexity of synthetic data. To tackle these
challenges, we introduce FakeVLM, a specialized large multimodal model designed
for both general synthetic image and DeepFake detection tasks. FakeVLM not only
excels in distinguishing real from fake images but also provides clear, natural
language explanations for image artifacts, enhancing interpretability.
Additionally, we present FakeClue, a comprehensive dataset containing over
100,000 images across seven categories, annotated with fine-grained artifact
clues in natural language. FakeVLM demonstrates performance comparable to
expert models while eliminating the need for additional classifiers, making it
a robust solution for synthetic data detection. Extensive evaluations across
multiple datasets confirm the superiority of FakeVLM in both authenticity
classification and artifact explanation tasks, setting a new benchmark for
synthetic image detection. The dataset and code will be released in:
https://github.com/opendatalab/FakeVLM.
MDocAgent: A Multi-Modal Multi-Agent Framework for Document
  Understanding
Authors: name: Siwei Han, name: Peng Xia, name: Ruiyi Zhang, name: Tong Sun, name: Yun Li, name: Hongtu Zhu, name: Huaxiu Yao
Published: 2025-03-18 06:57:21

Document Question Answering (DocQA) is a very common task. Existing methods
using Large Language Models (LLMs) or Large Vision Language Models (LVLMs) and
Retrieval Augmented Generation (RAG) often prioritize information from a single
modal, failing to effectively integrate textual and visual cues. These
approaches struggle with complex multi-modal reasoning, limiting their
performance on real-world documents. We present MDocAgent (A Multi-Modal
Multi-Agent Framework for Document Understanding), a novel RAG and multi-agent
framework that leverages both text and image. Our system employs five
specialized agents: a general agent, a critical agent, a text agent, an image
agent and a summarizing agent. These agents engage in multi-modal context
retrieval, combining their individual insights to achieve a more comprehensive
understanding of the document's content. This collaborative approach enables
the system to synthesize information from both textual and visual components,
leading to improved accuracy in question answering. Preliminary experiments on
five benchmarks like MMLongBench, LongDocURL demonstrate the effectiveness of
our MDocAgent, achieve an average improvement of 12.1% compared to current
state-of-the-art method. This work contributes to the development of more
robust and comprehensive DocQA systems capable of handling the complexities of
real-world documents containing rich textual and visual information. Our data
and code are available at https://github.com/aiming-lab/MDocAgent.
Scaling Vision Pre-Training to 4K Resolution
Authors: name: Baifeng Shi, name: Boyi Li, name: Han Cai, name: Yao Lu, name: Sifei Liu, name: Marco Pavone, name: Jan Kautz, name: Song Han, name: Trevor Darrell, name: Pavlo Molchanov, name: Hongxu Yin
Published: 2025-03-25 17:58:37

High-resolution perception of visual details is crucial for daily tasks.
Current vision pre-training, however, is still limited to low resolutions
(e.g., 378 x 378 pixels) due to the quadratic cost of processing larger images.
We introduce PS3 that scales CLIP-style vision pre-training to 4K resolution
with a near-constant cost. Instead of contrastive learning on global image
representation, PS3 is pre-trained by selectively processing local regions and
contrasting them with local detailed captions, enabling high-resolution
representation learning with greatly reduced computational overhead. The
pre-trained PS3 is able to both encode the global image at low resolution and
selectively process local high-resolution regions based on their saliency or
relevance to a text prompt. When applying PS3 to multi-modal LLM (MLLM), the
resulting model, named VILA-HD, significantly improves high-resolution visual
perception compared to baselines without high-resolution vision pre-training
such as AnyRes and S^2 while using up to 4.3x fewer tokens. PS3 also unlocks
appealing scaling properties of VILA-HD, including scaling up resolution for
free and scaling up test-time compute for better performance. Compared to state
of the arts, VILA-HD outperforms previous MLLMs such as NVILA and Qwen2-VL
across multiple benchmarks and achieves better efficiency than latest token
pruning approaches. Finally, we find current benchmarks do not require
4K-resolution perception, which motivates us to propose 4KPro, a new benchmark
of image QA at 4K resolution, on which VILA-HD outperforms all previous MLLMs,
including a 14.5% improvement over GPT-4o, and a 3.2% improvement and 2.96x
speedup over Qwen2-VL.
Exploring Hallucination of Large Multimodal Models in Video
  Understanding: Benchmark, Analysis and Mitigation
Authors: name: Hongcheng Gao, name: Jiashu Qu, name: Jingyi Tang, name: Baolong Bi, name: Yue Liu, name: Hongyu Chen, name: Li Liang, name: Li Su, name: Qingming Huang
Published: 2025-03-25 13:12:17

The hallucination of large multimodal models (LMMs), providing responses that
appear correct but are actually incorrect, limits their reliability and
applicability. This paper aims to study the hallucination problem of LMMs in
video modality, which is dynamic and more challenging compared to static
modalities like images and text. From this motivation, we first present a
comprehensive benchmark termed HAVEN for evaluating hallucinations of LMMs in
video understanding tasks. It is built upon three dimensions, i.e.,
hallucination causes, hallucination aspects, and question formats, resulting in
6K questions. Then, we quantitatively study 7 influential factors on
hallucinations, e.g., duration time of videos, model sizes, and model
reasoning, via experiments of 16 LMMs on the presented benchmark. In addition,
inspired by recent thinking models like OpenAI o1, we propose a video-thinking
model to mitigate the hallucinations of LMMs via supervised reasoning
fine-tuning (SRFT) and direct preference optimization (TDPO)-- where SRFT
enhances reasoning capabilities while TDPO reduces hallucinations in the
thinking process. Extensive experiments and analyses demonstrate the
effectiveness. Remarkably, it improves the baseline by 7.65% in accuracy on
hallucination evaluation and reduces the bias score by 4.5%. The code and data
are public at https://github.com/Hongcheng-Gao/HAVEN.
CoMP: Continual Multimodal Pre-training for Vision Foundation Models
Authors: name: Yitong Chen, name: Lingchen Meng, name: Wujian Peng, name: Zuxuan Wu, name: Yu-Gang Jiang
Published: 2025-03-24 17:52:47

Pre-trained Vision Foundation Models (VFMs) provide strong visual
representations for a wide range of applications. In this paper, we continually
pre-train prevailing VFMs in a multimodal manner such that they can
effortlessly process visual inputs of varying sizes and produce visual
representations that are more aligned with language representations, regardless
of their original pre-training process. To this end, we introduce CoMP, a
carefully designed multimodal pre-training pipeline. CoMP uses a Continual
Rotary Position Embedding to support native resolution continual pre-training,
and an Alignment Loss between visual and textual features through language
prototypes to align multimodal representations. By three-stage training, our
VFMs achieve remarkable improvements not only in multimodal understanding but
also in other downstream tasks such as classification and segmentation.
Remarkably, CoMP-SigLIP achieves scores of 66.7 on ChartQA and 75.9 on DocVQA
with a 0.5B LLM, while maintaining an 87.4% accuracy on ImageNet-1K and a 49.5
mIoU on ADE20K under frozen chunk evaluation.
Frequency Dynamic Convolution for Dense Image Prediction
Authors: name: Linwei Chen, name: Lin Gu, name: Liang Li, name: Chenggang Yan, name: Ying Fu
Published: 2025-03-24 15:32:06

While Dynamic Convolution (DY-Conv) has shown promising performance by
enabling adaptive weight selection through multiple parallel weights combined
with an attention mechanism, the frequency response of these weights tends to
exhibit high similarity, resulting in high parameter costs but limited
adaptability. In this work, we introduce Frequency Dynamic Convolution
(FDConv), a novel approach that mitigates these limitations by learning a fixed
parameter budget in the Fourier domain. FDConv divides this budget into
frequency-based groups with disjoint Fourier indices, enabling the construction
of frequency-diverse weights without increasing the parameter cost. To further
enhance adaptability, we propose Kernel Spatial Modulation (KSM) and Frequency
Band Modulation (FBM). KSM dynamically adjusts the frequency response of each
filter at the spatial level, while FBM decomposes weights into distinct
frequency bands in the frequency domain and modulates them dynamically based on
local content. Extensive experiments on object detection, segmentation, and
classification validate the effectiveness of FDConv. We demonstrate that when
applied to ResNet-50, FDConv achieves superior performance with a modest
increase of +3.6M parameters, outperforming previous methods that require
substantial increases in parameter budgets (e.g., CondConv +90M, KW +76.5M).
Moreover, FDConv seamlessly integrates into a variety of architectures,
including ConvNeXt, Swin-Transformer, offering a flexible and efficient
solution for modern vision tasks. The code is made publicly available at
https://github.com/Linwei-Chen/FDConv.
Latent Space Super-Resolution for Higher-Resolution Image Generation
  with Diffusion Models
Authors: name: Jinho Jeong, name: Sangmin Han, name: Jinwoo Kim, name: Seon Joo Kim
Published: 2025-03-24 08:50:15

In this paper, we propose LSRNA, a novel framework for higher-resolution
(exceeding 1K) image generation using diffusion models by leveraging
super-resolution directly in the latent space. Existing diffusion models
struggle with scaling beyond their training resolutions, often leading to
structural distortions or content repetition. Reference-based methods address
the issues by upsampling a low-resolution reference to guide higher-resolution
generation. However, they face significant challenges: upsampling in latent
space often causes manifold deviation, which degrades output quality. On the
other hand, upsampling in RGB space tends to produce overly smoothed outputs.
To overcome these limitations, LSRNA combines Latent space Super-Resolution
(LSR) for manifold alignment and Region-wise Noise Addition (RNA) to enhance
high-frequency details. Our extensive experiments demonstrate that integrating
LSRNA outperforms state-of-the-art reference-based methods across various
resolutions and metrics, while showing the critical role of latent space
upsampling in preserving detail and sharpness. The code is available at
https://github.com/3587jjh/LSRNA.
Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time
  Thinking
Authors: name: Xiaoyu Tian, name: Sitong Zhao, name: Haotian Wang, name: Shuaiting Chen, name: Yunjie Ji, name: Yiping Peng, name: Han Zhao, name: Xiangang Li
Published: 2025-03-25 17:19:38

Recent advances in large language models (LLMs), such as OpenAI-o1 and
DeepSeek-R1, have demonstrated the effectiveness of test-time scaling, where
extended reasoning processes substantially enhance model performance. Despite
this, current models are constrained by limitations in handling long texts and
reinforcement learning (RL) training efficiency. To address these issues, we
propose a simple yet effective test-time scaling approach Multi-round Thinking.
This method iteratively refines model reasoning by leveraging previous answers
as prompts for subsequent rounds. Extensive experiments across multiple models,
including QwQ-32B and DeepSeek-R1, consistently show performance improvements
on various benchmarks such as AIME 2024, MATH-500, GPQA-diamond, and
LiveCodeBench. For instance, the accuracy of QwQ-32B improved from 80.3% (Round
1) to 82.1% (Round 2) on the AIME 2024 dataset, while DeepSeek-R1 showed a
similar increase from 79.7% to 82.0%. These results confirm that Multi-round
Thinking is a broadly applicable, straightforward approach to achieving stable
enhancements in model performance, underscoring its potential for future
developments in test-time scaling techniques. The key prompt: {Original
question prompt} The assistant's previous answer is: <answer> {last round
answer} </answer>, and please re-answer.
ReSearch: Learning to Reason with Search for LLMs via Reinforcement
  Learning
Authors: name: Mingyang Chen, name: Tianpeng Li, name: Haoze Sun, name: Yijie Zhou, name: Chenzheng Zhu, name: Fan Yang, name: Zenan Zhou, name: Weipeng Chen, name: Haofen Wang, name: Jeff Z. Pan, name: Wen Zhang, name: Huajun Chen
Published: 2025-03-25 09:00:58

Large Language Models (LLMs) have shown remarkable capabilities in reasoning,
exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating
reasoning with external search processes remains challenging, especially for
complex multi-hop questions requiring multiple retrieval steps. We propose
ReSearch, a novel framework that trains LLMs to Reason with Search via
reinforcement learning without using any supervised data on reasoning steps.
Our approach treats search operations as integral components of the reasoning
chain, where when and how to perform searches is guided by text-based thinking,
and search results subsequently influence further reasoning. We train ReSearch
on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct
extensive experiments. Despite being trained on only one dataset, our models
demonstrate strong generalizability across various benchmarks. Analysis reveals
that ReSearch naturally elicits advanced reasoning capabilities such as
reflection and self-correction during the reinforcement learning process.
Inference-Time Scaling for Flow Models via Stochastic Generation and
  Rollover Budget Forcing
Authors: name: Jaihoon Kim, name: Taehoon Yoon, name: Jisung Hwang, name: Minhyuk Sung
Published: 2025-03-25 06:30:45

We propose an inference-time scaling approach for pretrained flow models.
Recently, inference-time scaling has gained significant attention in LLMs and
diffusion models, improving sample quality or better aligning outputs with user
preferences by leveraging additional computation. For diffusion models,
particle sampling has allowed more efficient scaling due to the stochasticity
at intermediate denoising steps. On the contrary, while flow models have gained
popularity as an alternative to diffusion models--offering faster generation
and high-quality outputs in state-of-the-art image and video generative
models--efficient inference-time scaling methods used for diffusion models
cannot be directly applied due to their deterministic generative process. To
enable efficient inference-time scaling for flow models, we propose three key
ideas: 1) SDE-based generation, enabling particle sampling in flow models, 2)
Interpolant conversion, broadening the search space and enhancing sample
diversity, and 3) Rollover Budget Forcing (RBF), an adaptive allocation of
computational resources across timesteps to maximize budget utilization. Our
experiments show that SDE-based generation, particularly variance-preserving
(VP) interpolant-based generation, improves the performance of particle
sampling methods for inference-time scaling in flow models. Additionally, we
demonstrate that RBF with VP-SDE achieves the best performance, outperforming
all previous inference-time scaling approaches.
Long-Context Autoregressive Video Modeling with Next-Frame Prediction
Authors: name: Yuchao Gu, name: Weijia Mao, name: Mike Zheng Shou
Published: 2025-03-25 03:38:06

Long-context autoregressive modeling has significantly advanced language
generation, but video generation still struggles to fully utilize extended
temporal contexts. To investigate long-context video modeling, we introduce
Frame AutoRegressive (FAR), a strong baseline for video autoregressive
modeling. Just as language models learn causal dependencies between tokens
(i.e., Token AR), FAR models temporal causal dependencies between continuous
frames, achieving better convergence than Token AR and video diffusion
transformers. Building on FAR, we observe that long-context vision modeling
faces challenges due to visual redundancy. Existing RoPE lacks effective
temporal decay for remote context and fails to extrapolate well to long video
sequences. Additionally, training on long videos is computationally expensive,
as vision tokens grow much faster than language tokens. To tackle these issues,
we propose balancing locality and long-range dependency. We introduce FlexRoPE,
an test-time technique that adds flexible temporal decay to RoPE, enabling
extrapolation to 16x longer vision contexts. Furthermore, we propose long
short-term context modeling, where a high-resolution short-term context window
ensures fine-grained temporal consistency, while an unlimited long-term context
window encodes long-range information using fewer tokens. With this approach,
we can train on long video sequences with a manageable token context length. We
demonstrate that FAR achieves state-of-the-art performance in both short- and
long-video generation, providing a simple yet effective baseline for video
autoregressive modeling.