Orionnnn/hub-stats huggingface dataset:
MetaUAS: Universal Anomaly Segmentation with One-Prompt Meta-Learning
Authors: name: Bin-Bin Gao
Published: 2025-05-14 10:25:26

Zero- and few-shot visual anomaly segmentation relies on powerful
vision-language models that detect unseen anomalies using manually designed
textual prompts. However, visual representations are inherently independent of
language. In this paper, we explore the potential of a pure visual foundation
model as an alternative to widely used vision-language models for universal
visual anomaly segmentation. We present a novel paradigm that unifies anomaly
segmentation into change segmentation. This paradigm enables us to leverage
large-scale synthetic image pairs, featuring object-level and local region
changes, derived from existing image datasets, which are independent of target
anomaly datasets. We propose a one-prompt Meta-learning framework for Universal
Anomaly Segmentation (MetaUAS) that is trained on this synthetic dataset and
then generalizes well to segment any novel or unseen visual anomalies in the
real world. To handle geometrical variations between prompt and query images,
we propose a soft feature alignment module that bridges paired-image change
perception and single-image semantic segmentation. This is the first work to
achieve universal anomaly segmentation using a pure vision model without
relying on special anomaly detection datasets and pre-trained visual-language
models. Our method effectively and efficiently segments any anomalies with only
one normal image prompt and enjoys training-free without guidance from
language. Our MetaUAS significantly outperforms previous zero-shot, few-shot,
and even full-shot anomaly segmentation methods. The code and pre-trained
models are available at https://github.com/gaobb/MetaUAS.
AdaptCLIP: Adapting CLIP for Universal Visual Anomaly Detection
Authors: name: Bin-Bin Gao, name: Yue Zhu, name: Jiangtao Yan, name: Yuezhi Cai, name: Weixi Zhang, name: Meng Wang, name: Jun Liu, name: Yong Liu, name: Lei Wang, name: Chengjie Wang
Published: 2025-05-15 03:24:28

Universal visual anomaly detection aims to identify anomalies from novel or
unseen vision domains without additional fine-tuning, which is critical in open
scenarios. Recent studies have demonstrated that pre-trained vision-language
models like CLIP exhibit strong generalization with just zero or a few normal
images. However, existing methods struggle with designing prompt templates,
complex token interactions, or requiring additional fine-tuning, resulting in
limited flexibility. In this work, we present a simple yet effective method
called AdaptCLIP based on two key insights. First, adaptive visual and textual
representations should be learned alternately rather than jointly. Second,
comparative learning between query and normal image prompt should incorporate
both contextual and aligned residual features, rather than relying solely on
residual features. AdaptCLIP treats CLIP models as a foundational service,
adding only three simple adapters, visual adapter, textual adapter, and
prompt-query adapter, at its input or output ends. AdaptCLIP supports
zero-/few-shot generalization across domains and possesses a training-free
manner on target domains once trained on a base dataset. AdaptCLIP achieves
state-of-the-art performance on 12 anomaly detection benchmarks from industrial
and medical domains, significantly outperforming existing competitive methods.
We will make the code and model of AdaptCLIP available at
https://github.com/gaobb/AdaptCLIP.
MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal
  Mathematical Reasoning
Authors: name: Ke Wang, name: Junting Pan, name: Linda Wei, name: Aojun Zhou, name: Weikang Shi, name: Zimu Lu, name: Han Xiao, name: Yunqiao Yang, name: Houxing Ren, name: Mingjie Zhan, name: Hongsheng Li
Published: 2025-05-15 17:59:21

Natural language image-caption datasets, widely used for training Large
Multimodal Models, mainly focus on natural scenarios and overlook the intricate
details of mathematical figures that are critical for problem-solving,
hindering the advancement of current LMMs in multimodal mathematical reasoning.
To this end, we propose leveraging code as supervision for cross-modal
alignment, since code inherently encodes all information needed to generate
corresponding figures, establishing a precise connection between the two
modalities. Specifically, we co-develop our image-to-code model and dataset
with model-in-the-loop approach, resulting in an image-to-code model,
FigCodifier and ImgCode-8.6M dataset, the largest image-code dataset to date.
Furthermore, we utilize FigCodifier to synthesize novel mathematical figures
and then construct MM-MathInstruct-3M, a high-quality multimodal math
instruction fine-tuning dataset. Finally, we present MathCoder-VL, trained with
ImgCode-8.6M for cross-modal alignment and subsequently fine-tuned on
MM-MathInstruct-3M for multimodal math problem solving. Our model achieves a
new open-source SOTA across all six metrics. Notably, it surpasses GPT-4o and
Claude 3.5 Sonnet in the geometry problem-solving subset of MathVista,
achieving improvements of 8.9% and 9.2%. The dataset and models will be
released at https://github.com/mathllm/MathCoder.
LightLab: Controlling Light Sources in Images with Diffusion Models
Authors: name: Nadav Magar, name: Amir Hertz, name: Eric Tabellion, name: Yael Pritch, name: Alex Rav-Acha, name: Ariel Shamir, name: Yedid Hoshen
Published: 2025-05-14 17:57:27

We present a simple, yet effective diffusion-based method for fine-grained,
parametric control over light sources in an image. Existing relighting methods
either rely on multiple input views to perform inverse rendering at inference
time, or fail to provide explicit control over light changes. Our method
fine-tunes a diffusion model on a small set of real raw photograph pairs,
supplemented by synthetically rendered images at scale, to elicit its
photorealistic prior for relighting. We leverage the linearity of light to
synthesize image pairs depicting controlled light changes of either a target
light source or ambient illumination. Using this data and an appropriate
fine-tuning scheme, we train a model for precise illumination changes with
explicit control over light intensity and color. Lastly, we show how our method
can achieve compelling light editing results, and outperforms existing methods
based on user preference.
Understanding and Mitigating Toxicity in Image-Text Pretraining
  Datasets: A Case Study on LLaVA
Authors: name: Karthik Reddy Kanjula, name: Surya Guthikonda, name: Nahid Alam, name: Shayekh Bin Islam
Published: 2025-05-09 18:01:50

Pretraining datasets are foundational to the development of multimodal
models, yet they often have inherent biases and toxic content from the
web-scale corpora they are sourced from. In this paper, we investigate the
prevalence of toxicity in LLaVA image-text pretraining dataset, examining how
harmful content manifests in different modalities. We present a comprehensive
analysis of common toxicity categories and propose targeted mitigation
strategies, resulting in the creation of a refined toxicity-mitigated dataset.
This dataset removes 7,531 of toxic image-text pairs in the LLaVA pre-training
dataset. We offer guidelines for implementing robust toxicity detection
pipelines. Our findings underscore the need to actively identify and filter
toxic content - such as hate speech, explicit imagery, and targeted harassment
- to build more responsible and equitable multimodal systems. The
toxicity-mitigated dataset is open source and is available for further
research.
Behind Maya: Building a Multilingual Vision Language Model
Authors: name: Nahid Alam, name: Karthik Reddy Kanjula, name: Surya Guthikonda, name: Timothy Chung, name: Bala Krishna S Vegesna, name: Abhipsha Das, name: Anthony Susevski, name: Ryan Sze-Yin Chan, name: S M Iftekhar Uddin, name: Shayekh Bin Islam, name: Roshan Santhosh, name: Snegha A, name: Drishti Sharma, name: Chen Liu, name: Isha Chaturvedi, name: Genta Indra Winata, name: Ashvanth. S, name: Snehanshu Mukherjee, name: Alham Fikri Aji
Published: 2025-05-13 19:01:12

In recent times, we have seen a rapid development of large Vision-Language
Models (VLMs). They have shown impressive results on academic benchmarks,
primarily in widely spoken languages but lack performance on low-resource
languages and varied cultural contexts. To address these limitations, we
introduce Maya, an open-source Multilingual VLM. Our contributions are: 1) a
multilingual image-text pretraining dataset in eight languages, based on the
LLaVA pretraining dataset; and 2) a multilingual image-text model supporting
these languages, enhancing cultural and linguistic comprehension in
vision-language tasks. Code available at https://github.com/nahidalam/maya.
Steepest Descent Density Control for Compact 3D Gaussian Splatting
Authors: name: Peihao Wang, name: Yuehao Wang, name: Dilin Wang, name: Sreyas Mohan, name: Zhiwen Fan, name: Lemeng Wu, name: Ruisi Cai, name: Yu-Ying Yeh, name: Zhangyang Wang, name: Qiang Liu, name: Rakesh Ranjan
Published: 2025-05-08 18:41:38

3D Gaussian Splatting (3DGS) has emerged as a powerful technique for
real-time, high-resolution novel view synthesis. By representing scenes as a
mixture of Gaussian primitives, 3DGS leverages GPU rasterization pipelines for
efficient rendering and reconstruction. To optimize scene coverage and capture
fine details, 3DGS employs a densification algorithm to generate additional
points. However, this process often leads to redundant point clouds, resulting
in excessive memory usage, slower performance, and substantial storage demands
- posing significant challenges for deployment on resource-constrained devices.
To address this limitation, we propose a theoretical framework that demystifies
and improves density control in 3DGS. Our analysis reveals that splitting is
crucial for escaping saddle points. Through an optimization-theoretic approach,
we establish the necessary conditions for densification, determine the minimal
number of offspring Gaussians, identify the optimal parameter update direction,
and provide an analytical solution for normalizing off-spring opacity. Building
on these insights, we introduce SteepGS, incorporating steepest density
control, a principled strategy that minimizes loss while maintaining a compact
point cloud. SteepGS achieves a ~50% reduction in Gaussian points without
compromising rendering quality, significantly enhancing both efficiency and
scalability.
WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
Authors: name: Shengpeng Ji, name: Tianle Liang, name: Yangzhuo Li, name: Jialong Zuo, name: Minghui Fang, name: Jinzheng He, name: Yifu Chen, name: Zhengqing Liu, name: Ziyue Jiang, name: Xize Cheng, name: Siqi Zheng, name: Jin Xu, name: Junyang Lin, name: Zhou Zhao
Published: 2025-05-14 16:54:15

End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered
significant attention in the speech domain. However, the evaluation of spoken
dialogue models' conversational performance has largely been overlooked. This
is primarily due to the intelligent chatbots convey a wealth of non-textual
information which cannot be easily measured using text-based language models
like ChatGPT. To address this gap, we propose WavReward, a reward feedback
model based on audio language models that can evaluate both the IQ and EQ of
spoken dialogue systems with speech input. Specifically, 1) based on audio
language models, WavReward incorporates the deep reasoning process and the
nonlinear reward mechanism for post-training. By utilizing multi-sample
feedback via the reinforcement learning algorithm, we construct a specialized
evaluator tailored to spoken dialogue models. 2) We introduce ChatReward-30K, a
preference dataset used to train WavReward. ChatReward-30K includes both
comprehension and generation aspects of spoken dialogue models. These scenarios
span various tasks, such as text-based chats, nine acoustic attributes of
instruction chats, and implicit chats. WavReward outperforms previous
state-of-the-art evaluation models across multiple spoken dialogue scenarios,
achieving a substantial improvement about Qwen2.5-Omni in objective accuracy
from 55.1% to 91.5%. In subjective A/B testing, WavReward also leads by a
margin of 83%. Comprehensive ablation studies confirm the necessity of each
component of WavReward. All data and code will be publicly at
https://github.com/jishengpeng/WavReward after the paper is accepted.
UniSkill: Imitating Human Videos via Cross-Embodiment Skill
  Representations
Authors: name: Hanjung Kim, name: Jaehyun Kang, name: Hyolim Kang, name: Meedeum Cho, name: Seon Joo Kim, name: Youngwoon Lee
Published: 2025-05-13 17:59:22

Mimicry is a fundamental learning mechanism in humans, enabling individuals
to learn new tasks by observing and imitating experts. However, applying this
ability to robots presents significant challenges due to the inherent
differences between human and robot embodiments in both their visual appearance
and physical capabilities. While previous methods bridge this gap using
cross-embodiment datasets with shared scenes and tasks, collecting such aligned
data between humans and robots at scale is not trivial. In this paper, we
propose UniSkill, a novel framework that learns embodiment-agnostic skill
representations from large-scale cross-embodiment video data without any
labels, enabling skills extracted from human video prompts to effectively
transfer to robot policies trained only on robot data. Our experiments in both
simulation and real-world environments show that our cross-embodiment skills
successfully guide robots in selecting appropriate actions, even with unseen
video prompts. The project website can be found at:
https://kimhanjung.github.io/UniSkill.
Visually Interpretable Subtask Reasoning for Visual Question Answering
Authors: name: Yu Cheng, name: Arushi Goel, name: Hakan Bilen
Published: 2025-05-12 21:37:06

Answering complex visual questions like `Which red furniture can be used for
sitting?' requires multi-step reasoning, including object recognition,
attribute filtering, and relational understanding. Recent work improves
interpretability in multimodal large language models (MLLMs) by decomposing
tasks into sub-task programs, but these methods are computationally expensive
and less accurate due to poor adaptation to target data. To address this, we
introduce VISTAR (Visually Interpretable Subtask-Aware Reasoning Model), a
subtask-driven training framework that enhances both interpretability and
reasoning by generating textual and visual explanations within MLLMs. Instead
of relying on external models, VISTAR fine-tunes MLLMs to produce structured
Subtask-of-Thought rationales (step-by-step reasoning sequences). Experiments
on two benchmarks show that VISTAR consistently improves reasoning accuracy
while maintaining interpretability. Our code and dataset will be available at
https://github.com/ChengJade/VISTAR.
CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image
Authors: name: Kaixin Yao, name: Longwen Zhang, name: Xinhao Yan, name: Yan Zeng, name: Qixuan Zhang, name: Lan Xu, name: Wei Yang, name: Jiayuan Gu, name: Jingyi Yu
Published: 2025-02-18 14:29:52

Recovering high-quality 3D scenes from a single RGB image is a challenging
task in computer graphics. Current methods often struggle with domain-specific
limitations or low-quality object generation. To address these, we propose CAST
(Component-Aligned 3D Scene Reconstruction from a Single RGB Image), a novel
method for 3D scene reconstruction and recovery. CAST starts by extracting
object-level 2D segmentation and relative depth information from the input
image, followed by using a GPT-based model to analyze inter-object spatial
relationships. This enables the understanding of how objects relate to each
other within the scene, ensuring more coherent reconstruction. CAST then
employs an occlusion-aware large-scale 3D generation model to independently
generate each object's full geometry, using MAE and point cloud conditioning to
mitigate the effects of occlusions and partial object information, ensuring
accurate alignment with the source image's geometry and texture. To align each
object with the scene, the alignment generation model computes the necessary
transformations, allowing the generated meshes to be accurately placed and
integrated into the scene's point cloud. Finally, CAST incorporates a
physics-aware correction step that leverages a fine-grained relation graph to
generate a constraint graph. This graph guides the optimization of object
poses, ensuring physical consistency and spatial coherence. By utilizing Signed
Distance Fields (SDF), the model effectively addresses issues such as
occlusions, object penetration, and floating objects, ensuring that the
generated scene accurately reflects real-world physical interactions. CAST can
be leveraged in robotics, enabling efficient real-to-simulation workflows and
providing realistic, scalable simulation environments for robotic systems.
DetReIDX: A Stress-Test Dataset for Real-World UAV-Based Person
  Recognition
Authors: name: Kailash A. Hambarde, name: Nzakiese Mbongo, name: Pavan Kumar MP, name: Satish Mekewad, name: Carolina Fernandes, name: Gökhan Silahtaroğlu, name: Alice Nithya, name: Pawan Wasnik, name: MD. Rashidunnabi, name: Pranita Samale, name: Hugo Proença
Published: 2025-05-07 20:41:06

Person reidentification (ReID) technology has been considered to perform
relatively well under controlled, ground-level conditions, but it breaks down
when deployed in challenging real-world settings. Evidently, this is due to
extreme data variability factors such as resolution, viewpoint changes, scale
variations, occlusions, and appearance shifts from clothing or session drifts.
Moreover, the publicly available data sets do not realistically incorporate
such kinds and magnitudes of variability, which limits the progress of this
technology. This paper introduces DetReIDX, a large-scale aerial-ground person
dataset, that was explicitly designed as a stress test to ReID under real-world
conditions. DetReIDX is a multi-session set that includes over 13 million
bounding boxes from 509 identities, collected in seven university campuses from
three continents, with drone altitudes between 5.8 and 120 meters. More
important, as a key novelty, DetReIDX subjects were recorded in (at least) two
sessions on different days, with changes in clothing, daylight and location,
making it suitable to actually evaluate long-term person ReID. Plus, data were
annotated from 16 soft biometric attributes and multitask labels for detection,
tracking, ReID, and action recognition. In order to provide empirical evidence
of DetReIDX usefulness, we considered the specific tasks of human detection and
ReID, where SOTA methods catastrophically degrade performance (up to 80% in
detection accuracy and over 70% in Rank-1 ReID) when exposed to DetReIDXs
conditions. The dataset, annotations, and official evaluation protocols are
publicly available at https://www.it.ubi.pt/DetReIDX/
Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?
Authors: name: Andrew Rouditchenko, name: Saurabhchand Bhati, name: Edson Araujo, name: Samuel Thomas, name: Hilde Kuehne, name: Rogerio Feris, name: James Glass
Published: 2025-05-14 14:47:16

We propose Omni-R1 which fine-tunes a recent multi-modal LLM, Qwen2.5-Omni,
on an audio question answering dataset with the reinforcement learning method
GRPO. This leads to new State-of-the-Art performance on the recent MMAU
benchmark. Omni-R1 achieves the highest accuracies on the sounds, music,
speech, and overall average categories, both on the Test-mini and Test-full
splits. To understand the performance improvement, we tested models both with
and without audio and found that much of the performance improvement from GRPO
could be attributed to better text-based reasoning. We also made a surprising
discovery that fine-tuning without audio on a text-only dataset was effective
at improving the audio-based performance.
SweRank: Software Issue Localization with Code Ranking
Authors: name: Revanth Gangi Reddy, name: Tarun Suresh, name: JaeHyeok Doo, name: Ye Liu, name: Xuan Phi Nguyen, name: Yingbo Zhou, name: Semih Yavuz, name: Caiming Xiong, name: Heng Ji, name: Shafiq Joty
Published: 2025-05-07 19:44:09

Software issue localization, the task of identifying the precise code
locations (files, classes, or functions) relevant to a natural language issue
description (e.g., bug report, feature request), is a critical yet
time-consuming aspect of software development. While recent LLM-based agentic
approaches demonstrate promise, they often incur significant latency and cost
due to complex multi-step reasoning and relying on closed-source LLMs.
Alternatively, traditional code ranking models, typically optimized for
query-to-code or code-to-code retrieval, struggle with the verbose and
failure-descriptive nature of issue localization queries. To bridge this gap,
we introduce SweRank, an efficient and effective retrieve-and-rerank framework
for software issue localization. To facilitate training, we construct SweLoc, a
large-scale dataset curated from public GitHub repositories, featuring
real-world issue descriptions paired with corresponding code modifications.
Empirical results on SWE-Bench-Lite and LocBench show that SweRank achieves
state-of-the-art performance, outperforming both prior ranking models and
costly agent-based systems using closed-source LLMs like Claude-3.5. Further,
we demonstrate SweLoc's utility in enhancing various existing retriever and
reranker models for issue localization, establishing the dataset as a valuable
resource for the community.
Marigold: Affordable Adaptation of Diffusion-Based Image Generators for
  Image Analysis
Authors: name: Bingxin Ke, name: Kevin Qu, name: Tianfu Wang, name: Nando Metzger, name: Shengyu Huang, name: Bo Li, name: Anton Obukhov, name: Konrad Schindler
Published: 2025-05-14 13:07:03

The success of deep learning in computer vision over the past decade has
hinged on large labeled datasets and strong pretrained models. In data-scarce
settings, the quality of these pretrained models becomes crucial for effective
transfer learning. Image classification and self-supervised learning have
traditionally been the primary methods for pretraining CNNs and
transformer-based architectures. Recently, the rise of text-to-image generative
models, particularly those using denoising diffusion in a latent space, has
introduced a new class of foundational models trained on massive, captioned
image datasets. These models' ability to generate realistic images of unseen
content suggests they possess a deep understanding of the visual world. In this
work, we present Marigold, a family of conditional generative models and a
fine-tuning protocol that extracts the knowledge from pretrained latent
diffusion models like Stable Diffusion and adapts them for dense image analysis
tasks, including monocular depth estimation, surface normals prediction, and
intrinsic decomposition. Marigold requires minimal modification of the
pre-trained latent diffusion model's architecture, trains with small synthetic
datasets on a single GPU over a few days, and demonstrates state-of-the-art
zero-shot generalization. Project page:
https://marigoldcomputervision.github.io