Orionnnn/hub-stats huggingface dataset:
Boosting Generative Image Modeling via Joint Image-Feature Synthesis
Authors: name: Theodoros Kouzelis, name: Efstathios Karypidis, name: Ioannis Kakogeorgiou, name: Spyros Gidaris, name: Nikos Komodakis
Published: 2025-04-22 17:41:42

Latent diffusion models (LDMs) dominate high-quality image generation, yet
integrating representation learning with generative modeling remains a
challenge. We introduce a novel generative image modeling framework that
seamlessly bridges this gap by leveraging a diffusion model to jointly model
low-level image latents (from a variational autoencoder) and high-level
semantic features (from a pretrained self-supervised encoder like DINO). Our
latent-semantic diffusion approach learns to generate coherent image-feature
pairs from pure noise, significantly enhancing both generative quality and
training efficiency, all while requiring only minimal modifications to standard
Diffusion Transformer architectures. By eliminating the need for complex
distillation objectives, our unified design simplifies training and unlocks a
powerful new inference strategy: Representation Guidance, which leverages
learned semantics to steer and refine image generation. Evaluated in both
conditional and unconditional settings, our method delivers substantial
improvements in image quality and training convergence speed, establishing a
new direction for representation-aware generative modeling.
RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image
  Generation
Authors: name: Aviv Slobodkin, name: Hagai Taitelbaum, name: Yonatan Bitton, name: Brian Gordon, name: Michal Sokolik, name: Nitzan Bitton Guetta, name: Almog Gueta, name: Royi Rassin, name: Itay Laish, name: Dani Lischinski, name: Idan Szpektor
Published: 2025-04-24 12:44:51

Subject-driven text-to-image (T2I) generation aims to produce images that
align with a given textual description, while preserving the visual identity
from a referenced subject image. Despite its broad downstream applicability --
ranging from enhanced personalization in image generation to consistent
character representation in video rendering -- progress in this field is
limited by the lack of reliable automatic evaluation. Existing methods either
assess only one aspect of the task (i.e., textual alignment or subject
preservation), misalign with human judgments, or rely on costly API-based
evaluation. To address this, we introduce RefVNLI, a cost-effective metric that
evaluates both textual alignment and subject preservation in a single
prediction. Trained on a large-scale dataset derived from video-reasoning
benchmarks and image perturbations, RefVNLI outperforms or matches existing
baselines across multiple benchmarks and subject categories (e.g.,
Animal, Object), achieving up to 6.4-point gains in textual
alignment and 8.5-point gains in subject consistency. It also excels with
lesser-known concepts, aligning with human preferences at over 87\% accuracy.
ViSMaP: Unsupervised Hour-long Video Summarisation by Meta-Prompting
Authors: name: Jian Hu, name: Dimitrios Korkinof, name: Shaogang Gong, name: Mariano Beguerisse-Diaz
Published: 2025-04-22 14:06:01

We introduce ViSMap: Unsupervised Video Summarisation by Meta Prompting, a
system to summarise hour long videos with no-supervision. Most existing video
understanding models work well on short videos of pre-segmented events, yet
they struggle to summarise longer videos where relevant events are sparsely
distributed and not pre-segmented. Moreover, long-form video understanding
often relies on supervised hierarchical training that needs extensive
annotations which are costly, slow and prone to inconsistency. With ViSMaP we
bridge the gap between short videos (where annotated data is plentiful) and
long ones (where it's not). We rely on LLMs to create optimised
pseudo-summaries of long videos using segment descriptions from short ones.
These pseudo-summaries are used as training data for a model that generates
long-form video summaries, bypassing the need for expensive annotations of long
videos. Specifically, we adopt a meta-prompting strategy to iteratively
generate and refine creating pseudo-summaries of long videos. The strategy
leverages short clip descriptions obtained from a supervised short video model
to guide the summary. Each iteration uses three LLMs working in sequence: one
to generate the pseudo-summary from clip descriptions, another to evaluate it,
and a third to optimise the prompt of the generator. This iteration is
necessary because the quality of the pseudo-summaries is highly dependent on
the generator prompt, and varies widely among videos. We evaluate our summaries
extensively on multiple datasets; our results show that ViSMaP achieves
performance comparable to fully supervised state-of-the-art models while
generalising across domains without sacrificing performance. Code will be
released upon publication.
TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming
  Videos
Authors: name: Linli Yao, name: Yicheng Li, name: Yuancheng Wei, name: Lei Li, name: Shuhuai Ren, name: Yuanxin Liu, name: Kun Ouyang, name: Lean Wang, name: Shicheng Li, name: Sida Li, name: Lingpeng Kong, name: Qi Liu, name: Yuanxing Zhang, name: Xu Sun
Published: 2025-04-24 07:59:46

The rapid growth of online video platforms, particularly live streaming
services, has created an urgent need for real-time video understanding systems.
These systems must process continuous video streams and respond to user queries
instantaneously, presenting unique challenges for current Video Large Language
Models (VideoLLMs). While existing VideoLLMs excel at processing complete
videos, they face significant limitations in streaming scenarios due to their
inability to handle dense, redundant frames efficiently. We introduce
TimeChat-Online, a novel online VideoLLM that revolutionizes real-time video
interaction. At its core lies our innovative Differential Token Drop (DTD)
module, which addresses the fundamental challenge of visual redundancy in
streaming videos. Drawing inspiration from human visual perception's Change
Blindness phenomenon, DTD preserves meaningful temporal changes while filtering
out static, redundant content between frames. Remarkably, our experiments
demonstrate that DTD achieves an 82.8% reduction in video tokens while
maintaining 98% performance on StreamingBench, revealing that over 80% of
visual content in streaming videos is naturally redundant without requiring
language guidance. To enable seamless real-time interaction, we present
TimeChat-Online-139K, a comprehensive streaming video dataset featuring diverse
interaction patterns including backward-tracing, current-perception, and
future-responding scenarios. TimeChat-Online's unique Proactive Response
capability, naturally achieved through continuous monitoring of video scene
transitions via DTD, sets it apart from conventional approaches. Our extensive
evaluation demonstrates TimeChat-Online's superior performance on streaming
benchmarks (StreamingBench and OvOBench) and maintaining competitive results on
long-form video tasks such as Video-MME and MLVU.
IberBench: LLM Evaluation on Iberian Languages
Authors: name: José Ángel González, name: Ian Borrego Obrador, name: Álvaro Romo Herrero, name: Areg Mikael Sarvazyan, name: Mara Chinea-Ríos, name: Angelo Basile, name: Marc Franco-Salvador
Published: 2025-04-23 17:48:25

Large Language Models (LLMs) remain difficult to evaluate comprehensively,
particularly for languages other than English, where high-quality data is often
limited. Existing benchmarks and leaderboards are predominantly
English-centric, with only a few addressing other languages. These benchmarks
fall short in several key areas: they overlook the diversity of language
varieties, prioritize fundamental Natural Language Processing (NLP)
capabilities over tasks of industrial relevance, and are static. With these
aspects in mind, we present IberBench, a comprehensive and extensible benchmark
designed to assess LLM performance on both fundamental and industry-relevant
NLP tasks, in languages spoken across the Iberian Peninsula and Ibero-America.
IberBench integrates 101 datasets from evaluation campaigns and recent
benchmarks, covering 22 task categories such as sentiment and emotion analysis,
toxicity detection, and summarization. The benchmark addresses key limitations
in current evaluation practices, such as the lack of linguistic diversity and
static evaluation setups by enabling continual updates and community-driven
model and dataset submissions moderated by a committee of experts. We evaluate
23 LLMs ranging from 100 million to 14 billion parameters and provide empirical
insights into their strengths and limitations. Our findings indicate that (i)
LLMs perform worse on industry-relevant tasks than in fundamental ones, (ii)
performance is on average lower for Galician and Basque, (iii) some tasks show
results close to random, and (iv) in other tasks LLMs perform above random but
below shared task systems. IberBench offers open-source implementations for the
entire evaluation pipeline, including dataset normalization and hosting,
incremental evaluation of LLMs, and a publicly accessible leaderboard.
3DV-TON: Textured 3D-Guided Consistent Video Try-on via Diffusion Models
Authors: name: Min Wei, name: Chaohui Yu, name: Jingkai Zhou, name: Fan Wang
Published: 2025-04-24 10:12:40

Video try-on replaces clothing in videos with target garments. Existing
methods struggle to generate high-quality and temporally consistent results
when handling complex clothing patterns and diverse body poses. We present
3DV-TON, a novel diffusion-based framework for generating high-fidelity and
temporally consistent video try-on results. Our approach employs generated
animatable textured 3D meshes as explicit frame-level guidance, alleviating the
issue of models over-focusing on appearance fidelity at the expanse of motion
coherence. This is achieved by enabling direct reference to consistent garment
texture movements throughout video sequences. The proposed method features an
adaptive pipeline for generating dynamic 3D guidance: (1) selecting a keyframe
for initial 2D image try-on, followed by (2) reconstructing and animating a
textured 3D mesh synchronized with original video poses. We further introduce a
robust rectangular masking strategy that successfully mitigates artifact
propagation caused by leaking clothing information during dynamic human and
garment movements. To advance video try-on research, we introduce HR-VVT, a
high-resolution benchmark dataset containing 130 videos with diverse clothing
types and scenarios. Quantitative and qualitative results demonstrate our
superior performance over existing methods. The project page is at this link
https://2y7c3.github.io/3DV-TON/
DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs
Authors: name: Zhenhailong Wang, name: Senthil Purushwalkam, name: Caiming Xiong, name: Silvio Savarese, name: Heng Ji, name: Ran Xu
Published: 2025-04-23 18:38:18

We present DyMU, an efficient, training-free framework that dynamically
reduces the computational burden of vision-language models (VLMs) while
maintaining high task performance. Our approach comprises two key components.
First, Dynamic Token Merging (DToMe) reduces the number of visual token
embeddings by merging similar tokens based on image complexity, addressing the
inherent inefficiency of fixed-length outputs in vision transformers. Second,
Virtual Token Unmerging (VTU) simulates the expected token sequence for large
language models (LLMs) by efficiently reconstructing the attention dynamics of
a full sequence, thus preserving the downstream performance without additional
fine-tuning. Unlike previous approaches, our method dynamically adapts token
compression to the content of the image and operates completely training-free,
making it readily applicable to most state-of-the-art VLM architectures.
Extensive experiments on image and video understanding tasks demonstrate that
DyMU can reduce the average visual token count by 32%-85% while achieving
comparable performance to full-length models across diverse VLM architectures,
including the recently popularized AnyRes-based visual encoders. Furthermore,
through qualitative analyses, we demonstrate that DToMe effectively adapts
token reduction based on image complexity and, unlike existing systems,
provides users more control over computational costs. Project page:
https://mikewangwzhl.github.io/dymu/.
Token-Shuffle: Towards High-Resolution Image Generation with
  Autoregressive Models
Authors: name: Xu Ma, name: Peize Sun, name: Haoyu Ma, name: Hao Tang, name: Chih-Yao Ma, name: Jialiang Wang, name: Kunpeng Li, name: Xiaoliang Dai, name: Yujun Shi, name: Xuan Ju, name: Yushi Hu, name: Artsiom Sanakoyeu, name: Felix Juefei-Xu, name: Ji Hou, name: Junjiao Tian, name: Tao Xu, name: Tingbo Hou, name: Yen-Cheng Liu, name: Zecheng He, name: Zijian He, name: Matt Feiszli, name: Peizhao Zhang, name: Peter Vajda, name: Sam Tsai, name: Yun Fu
Published: 2025-04-24 17:59:56

Autoregressive (AR) models, long dominant in language generation, are
increasingly applied to image synthesis but are often considered less
competitive than Diffusion-based models. A primary limitation is the
substantial number of image tokens required for AR models, which constrains
both training and inference efficiency, as well as image resolution. To address
this, we present Token-Shuffle, a novel yet simple method that reduces the
number of image tokens in Transformer. Our key insight is the dimensional
redundancy of visual vocabularies in Multimodal Large Language Models (MLLMs),
where low-dimensional visual codes from visual encoder are directly mapped to
high-dimensional language vocabularies. Leveraging this, we consider two key
operations: token-shuffle, which merges spatially local tokens along channel
dimension to decrease the input token number, and token-unshuffle, which
untangles the inferred tokens after Transformer blocks to restore the spatial
arrangement for output. Jointly training with textual prompts, our strategy
requires no additional pretrained text-encoder and enables MLLMs to support
extremely high-resolution image synthesis in a unified next-token prediction
way while maintaining efficient training and inference. For the first time, we
push the boundary of AR text-to-image generation to a resolution of 2048x2048
with gratifying generation performance. In GenAI-benchmark, our 2.7B model
achieves 0.77 overall score on hard prompts, outperforming AR models LlamaGen
by 0.18 and diffusion models LDM by 0.15. Exhaustive large-scale human
evaluations also demonstrate our prominent image generation ability in terms of
text-alignment, visual flaw, and visual appearance. We hope that Token-Shuffle
can serve as a foundational design for efficient high-resolution image
generation within MLLMs.
Interpretable non-linear dimensionality reduction using gaussian
  weighted linear transformation
Authors: name: Erik Bergh
Published: 2025-04-24 14:26:42

Dimensionality reduction techniques are fundamental for analyzing and
visualizing high-dimensional data. With established methods like t-SNE and PCA
presenting a trade-off between representational power and interpretability.
This paper introduces a novel approach that bridges this gap by combining the
interpretability of linear methods with the expressiveness of non-linear
transformations. The proposed algorithm constructs a non-linear mapping between
high-dimensional and low-dimensional spaces through a combination of linear
transformations, each weighted by Gaussian functions. This architecture enables
complex non-linear transformations while preserving the interpretability
advantages of linear methods, as each transformation can be analyzed
independently. The resulting model provides both powerful dimensionality
reduction and transparent insights into the transformed space. Techniques for
interpreting the learned transformations are presented, including methods for
identifying suppressed dimensions and how space is expanded and contracted.
These tools enable practitioners to understand how the algorithm preserves and
modifies geometric relationships during dimensionality reduction. To ensure the
practical utility of this algorithm, the creation of user-friendly software
packages is emphasized, facilitating its adoption in both academia and
industry.
QuaDMix: Quality-Diversity Balanced Data Selection for Efficient LLM
  Pretraining
Authors: name: Fengze Liu, name: Weidong Zhou, name: Binbin Liu, name: Zhimiao Yu, name: Yifan Zhang, name: Haobin Lin, name: Yifeng Yu, name: Xiaohuan Zhou, name: Taifeng Wang, name: Yong Cao
Published: 2025-04-23 08:36:50

Quality and diversity are two critical metrics for the training data of large
language models (LLMs), positively impacting performance. Existing studies
often optimize these metrics separately, typically by first applying quality
filtering and then adjusting data proportions. However, these approaches
overlook the inherent trade-off between quality and diversity, necessitating
their joint consideration. Given a fixed training quota, it is essential to
evaluate both the quality of each data point and its complementary effect on
the overall dataset. In this paper, we introduce a unified data selection
framework called QuaDMix, which automatically optimizes the data distribution
for LLM pretraining while balancing both quality and diversity. Specifically,
we first propose multiple criteria to measure data quality and employ domain
classification to distinguish data points, thereby measuring overall diversity.
QuaDMix then employs a unified parameterized data sampling function that
determines the sampling probability of each data point based on these quality
and diversity related labels. To accelerate the search for the optimal
parameters involved in the QuaDMix framework, we conduct simulated experiments
on smaller models and use LightGBM for parameters searching, inspired by the
RegMix method. Our experiments across diverse models and datasets demonstrate
that QuaDMix achieves an average performance improvement of 7.2% across
multiple benchmarks. These results outperform the independent strategies for
quality and diversity, highlighting the necessity and ability to balance data
quality and diversity.
Paper2Code: Automating Code Generation from Scientific Papers in Machine
  Learning
Authors: name: Minju Seo, name: Jinheon Baek, name: Seongyun Lee, name: Sung Ju Hwang
Published: 2025-04-24 01:57:01

Despite the rapid growth of machine learning research, corresponding code
implementations are often unavailable, making it slow and labor-intensive for
researchers to reproduce results and build upon prior work. In the meantime,
recent Large Language Models (LLMs) excel at understanding scientific documents
and generating high-quality code. Inspired by this, we introduce PaperCoder, a
multi-agent LLM framework that transforms machine learning papers into
functional code repositories. PaperCoder operates in three stages: planning,
where it constructs a high-level roadmap, designs the system architecture with
diagrams, identifies file dependencies, and generates configuration files;
analysis, which focuses on interpreting implementation-specific details; and
generation, where modular, dependency-aware code is produced. Moreover, each
phase is instantiated through a set of specialized agents designed to
collaborate effectively across the pipeline. We then evaluate PaperCoder on
generating code implementations from machine learning papers based on both
model-based and human evaluations, specifically from the original paper
authors, with author-released repositories as ground truth if available. Our
results demonstrate the effectiveness of PaperCoder in creating high-quality,
faithful implementations. Furthermore, it consistently shows strengths in the
recently released PaperBench benchmark, surpassing strong baselines by
substantial margins.
Distilling semantically aware orders for autoregressive image generation
Authors: name: Rishav Pramanik, name: Antoine Poupon, name: Juan A. Rodriguez, name: Masih Aminbeidokhti, name: David Vazquez, name: Christopher Pal, name: Zhaozheng Yin, name: Marco Pedersoli
Published: 2025-04-23 19:33:58

Autoregressive patch-based image generation has recently shown competitive
results in terms of image quality and scalability. It can also be easily
integrated and scaled within Vision-Language models. Nevertheless,
autoregressive models require a defined order for patch generation. While a
natural order based on the dictation of the words makes sense for text
generation, there is no inherent generation order that exists for image
generation. Traditionally, a raster-scan order (from top-left to bottom-right)
guides autoregressive image generation models. In this paper, we argue that
this order is suboptimal, as it fails to respect the causality of the image
content: for instance, when conditioned on a visual description of a sunset, an
autoregressive model may generate clouds before the sun, even though the color
of clouds should depend on the color of the sun and not the inverse. In this
work, we show that first by training a model to generate patches in
any-given-order, we can infer both the content and the location (order) of each
patch during generation. Secondly, we use these extracted orders to finetune
the any-given-order model to produce better-quality images. Through our
experiments, we show on two datasets that this new generation method produces
better images than the traditional raster-scan approach, with similar training
costs and no extra annotations.
Step1X-Edit: A Practical Framework for General Image Editing
Authors: name: Shiyu Liu, name: Yucheng Han, name: Peng Xing, name: Fukun Yin, name: Rui Wang, name: Wei Cheng, name: Jiaqi Liao, name: Yingming Wang, name: Honghao Fu, name: Chunrui Han, name: Guopeng Li, name: Yuang Peng, name: Quan Sun, name: Jingwei Wu, name: Yan Cai, name: Zheng Ge, name: Ranchen Ming, name: Lei Xia, name: Xianfang Zeng, name: Yibo Zhu, name: Binxing Jiao, name: Xiangyu Zhang, name: Gang Yu, name: Daxin Jiang
Published: 2025-04-24 17:25:12

In recent years, image editing models have witnessed remarkable and rapid
development. The recent unveiling of cutting-edge multimodal models such as
GPT-4o and Gemini2 Flash has introduced highly promising image editing
capabilities. These models demonstrate an impressive aptitude for fulfilling a
vast majority of user-driven editing requirements, marking a significant
advancement in the field of image manipulation. However, there is still a large
gap between the open-source algorithm with these closed-source models. Thus, in
this paper, we aim to release a state-of-the-art image editing model, called
Step1X-Edit, which can provide comparable performance against the closed-source
models like GPT-4o and Gemini2 Flash. More specifically, we adopt the
Multimodal LLM to process the reference image and the user's editing
instruction. A latent embedding has been extracted and integrated with a
diffusion image decoder to obtain the target image. To train the model, we
build a data generation pipeline to produce a high-quality dataset. For
evaluation, we develop the GEdit-Bench, a novel benchmark rooted in real-world
user instructions. Experimental results on GEdit-Bench demonstrate that
Step1X-Edit outperforms existing open-source baselines by a substantial margin
and approaches the performance of leading proprietary models, thereby making
significant contributions to the field of image editing.