Orionnnn/hub-stats huggingface dataset:
Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical
  Perspective
Authors: name: Weijie Xu, name: Yiwen Wang, name: Chi Xue, name: Xiangkun Hu, name: Xi Fang, name: Guimin Dong, name: Chandan K. Reddy
Published: 2025-06-23 18:31:22

Large Language Models (LLMs) often generate responses with inherent biases,
undermining their reliability in real-world applications. Existing evaluation
methods often overlook biases in long-form responses and the intrinsic
variability of LLM outputs. To address these challenges, we propose
FiSCo(Fine-grained Semantic Computation), a novel statistical framework to
evaluate group-level fairness in LLMs by detecting subtle semantic differences
in long-form responses across demographic groups. Unlike prior work focusing on
sentiment or token-level comparisons, FiSCo goes beyond surface-level analysis
by operating at the claim level, leveraging entailment checks to assess the
consistency of meaning across responses. We decompose model outputs into
semantically distinct claims and apply statistical hypothesis testing to
compare inter- and intra-group similarities, enabling robust detection of
subtle biases. We formalize a new group counterfactual fairness definition and
validate FiSCo on both synthetic and human-annotated datasets spanning gender,
race, and age. Experiments show that FiSco more reliably identifies nuanced
biases while reducing the impact of stochastic LLM variability, outperforming
various evaluation metrics.
RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language
  Models
Authors: name: Yeongtak Oh, name: Jisoo Mok, name: Dohyun Chung, name: Juhyeon Shin, name: Sangha Park, name: Johan Barthelemy, name: Sungroh Yoon
Published: 2025-06-23 07:55:52

Recent multi-modal large language models (MLLMs) often struggle to generate
personalized image captions, even when trained on high-quality captions. In
this work, we observe that such limitations persist in existing
post-training-based MLLM personalization methods. Specifically, despite being
post-tuned with large-scale caption data through supervised fine-tuning (SFT),
these models frequently fail to produce faithful descriptions in real-world
scenarios, such as multi-concept image captioning. However, acquiring
large-scale, high-quality captions for such complex settings is both costly and
difficult. To address the data-centric nature of SFT, we propose a
reinforcement learning (RL)-based post-training framework. To the best of our
knowledge, this is the first RL-based approach to post-train MLLMs for
personalized image captioning. Our method significantly enhances both visual
recognition and personalized generation capabilities of MLLMs, and consistently
outperforms existing SFT-based baselines, especially in the challenging
multi-concept image captioning task.
4D-LRM: Large Space-Time Reconstruction Model From and To Any View at
  Any Time
Authors: name: Ziqiao Ma, name: Xuweiyi Chen, name: Shoubin Yu, name: Sai Bi, name: Kai Zhang, name: Chen Ziwen, name: Sihan Xu, name: Jianing Yang, name: Zexiang Xu, name: Kalyan Sunkavalli, name: Mohit Bansal, name: Joyce Chai, name: Hao Tan
Published: 2025-06-23 17:57:47

Can we scale 4D pretraining to learn general space-time representations that
reconstruct an object from a few views at some times to any view at any time?
We provide an affirmative answer with 4D-LRM, the first large-scale 4D
reconstruction model that takes input from unconstrained views and timestamps
and renders arbitrary novel view-time combinations. Unlike prior 4D approaches,
e.g., optimization-based, geometry-based, or generative, that struggle with
efficiency, generalization, or faithfulness, 4D-LRM learns a unified space-time
representation and directly predicts per-pixel 4D Gaussian primitives from
posed image tokens across time, enabling fast, high-quality rendering at, in
principle, infinite frame rate. Our results demonstrate that scaling
spatiotemporal pretraining enables accurate and efficient 4D reconstruction. We
show that 4D-LRM generalizes to novel objects, interpolates across time, and
handles diverse camera setups. It reconstructs 24-frame sequences in one
forward pass with less than 1.5 seconds on a single A100 GPU.
Spec2RTL-Agent: Automated Hardware Code Generation from Complex
  Specifications Using LLM Agent Systems
Authors: name: Zhongzhi Yu, name: Mingjie Liu, name: Michael Zimmer, name: Yingyan Celine Lin, name: Yong Liu, name: Haoxing Ren
Published: 2025-06-16 18:33:25

Despite recent progress in generating hardware RTL code with LLMs, existing
solutions still suffer from a substantial gap between practical application
scenarios and the requirements of real-world RTL code development. Prior
approaches either focus on overly simplified hardware descriptions or depend on
extensive human guidance to process complex specifications, limiting their
scalability and automation potential. In this paper, we address this gap by
proposing an LLM agent system, termed Spec2RTL-Agent, designed to directly
process complex specification documentation and generate corresponding RTL code
implementations, advancing LLM-based RTL code generation toward more realistic
application settings. To achieve this goal, Spec2RTL-Agent introduces a novel
multi-agent collaboration framework that integrates three key enablers: (1) a
reasoning and understanding module that translates specifications into
structured, step-by-step implementation plans; (2) a progressive coding and
prompt optimization module that iteratively refines the code across multiple
representations to enhance correctness and synthesisability for RTL conversion;
and (3) an adaptive reflection module that identifies and traces the source of
errors during generation, ensuring a more robust code generation flow. Instead
of directly generating RTL from natural language, our system strategically
generates synthesizable C++ code, which is then optimized for HLS. This
agent-driven refinement ensures greater correctness and compatibility compared
to naive direct RTL generation approaches. We evaluate Spec2RTL-Agent on three
specification documents, showing it generates accurate RTL code with up to 75%
fewer human interventions than existing methods. This highlights its role as
the first fully automated multi-agent system for RTL generation from
unstructured specs, reducing reliance on human effort in hardware design.
Demystifying the Visual Quality Paradox in Multimodal Large Language
  Models
Authors: name: Shuo Xing, name: Lanqing Guo, name: Hongyuan Hua, name: Seoyoung Lee, name: Peiran Li, name: Yufei Wang, name: Zhangyang Wang, name: Zhengzhong Tu
Published: 2025-06-18 17:14:07

Recent Multimodal Large Language Models (MLLMs) excel on benchmark
vision-language tasks, yet little is known about how input visual quality
shapes their responses. Does higher perceptual quality of images already
translate to better MLLM understanding? We conduct the first systematic study
spanning leading MLLMs and a suite of vision-language benchmarks, applying
controlled degradations and stylistic shifts to each image. Surprisingly, we
uncover a visual-quality paradox: model, task, and even individual-instance
performance can improve when images deviate from human-perceived fidelity.
Off-the-shelf restoration pipelines fail to reconcile these idiosyncratic
preferences. To close the gap, we introduce Visual-Quality Test-Time Tuning
(VQ-TTT)-a lightweight adaptation module that: (1) inserts a learnable,
low-rank kernel before the frozen vision encoder to modulate frequency content;
and (2) fine-tunes only shallow vision-encoder layers via LoRA. VQ-TTT
dynamically adjusts each input image in a single forward pass, aligning it with
task-specific model preferences. Across the evaluated MLLMs and all datasets,
VQ-TTT lifts significant average accuracy, with no external models, cached
features, or extra training data. These findings redefine ``better'' visual
inputs for MLLMs and highlight the need for adaptive, rather than universally
``clean'', imagery, in the new era of AI being the main data customer.
3D Arena: An Open Platform for Generative 3D Evaluation
Authors: name: Dylan Ebert
Published: 2025-06-23 15:57:10

Evaluating Generative 3D models remains challenging due to misalignment
between automated metrics and human perception of quality. Current benchmarks
rely on image-based metrics that ignore 3D structure or geometric measures that
fail to capture perceptual appeal and real-world utility. To address this gap,
we present 3D Arena, an open platform for evaluating image-to-3D generation
models through large-scale human preference collection using pairwise
comparisons.
  Since launching in June 2024, the platform has collected 123,243 votes from
8,096 users across 19 state-of-the-art models, establishing the largest human
preference evaluation for Generative 3D. We contribute the iso3d dataset of 100
evaluation prompts and demonstrate quality control achieving 99.75% user
authenticity through statistical fraud detection. Our ELO-based ranking system
provides reliable model assessment, with the platform becoming an established
evaluation resource.
  Through analysis of this preference data, we present insights into human
preference patterns. Our findings reveal preferences for visual presentation
features, with Gaussian splat outputs achieving a 16.6 ELO advantage over
meshes and textured models receiving a 144.1 ELO advantage over untextured
models. We provide recommendations for improving evaluation methods, including
multi-criteria assessment, task-oriented evaluation, and format-aware
comparison. The platform's community engagement establishes 3D Arena as a
benchmark for the field while advancing understanding of human-centered
evaluation in Generative 3D.
Audit & Repair: An Agentic Framework for Consistent Story Visualization
  in Text-to-Image Diffusion Models
Authors: name: Kiymet Akdemir, name: Tahira Kazimi, name: Pinar Yanardag
Published: 2025-06-23 17:59:29

Story visualization has become a popular task where visual scenes are
generated to depict a narrative across multiple panels. A central challenge in
this setting is maintaining visual consistency, particularly in how characters
and objects persist and evolve throughout the story. Despite recent advances in
diffusion models, current approaches often fail to preserve key character
attributes, leading to incoherent narratives. In this work, we propose a
collaborative multi-agent framework that autonomously identifies, corrects, and
refines inconsistencies across multi-panel story visualizations. The agents
operate in an iterative loop, enabling fine-grained, panel-level updates
without re-generating entire sequences. Our framework is model-agnostic and
flexibly integrates with a variety of diffusion models, including rectified
flow transformers such as Flux and latent diffusion models such as Stable
Diffusion. Quantitative and qualitative experiments show that our method
outperforms prior approaches in terms of multi-panel consistency.
How Alignment Shrinks the Generative Horizon
Authors: name: Chenghao Yang, name: Ari Holtzman
Published: 2025-06-22 02:00:37

Despite their impressive capabilities, aligned large language models (LLMs)
often generate outputs that lack diversity. What drives this stability in the
generation? We investigate this phenomenon through the lens of probability
concentration in the model's output distribution. To quantify this
concentration, we introduce the Branching Factor (BF) -- a token-invariant
measure of the effective number of plausible next steps during generation. Our
empirical analysis reveals two key findings: (1) BF often decreases as
generation progresses, suggesting that LLMs become more predictable as they
generate. (2) alignment tuning substantially sharpens the model's output
distribution from the outset, reducing BF by nearly an order of magnitude
(e.g., from 12 to 1.2) relative to base models. This stark reduction helps
explain why aligned models often appear less sensitive to decoding strategies.
Building on this insight, we find this stability has surprising implications
for complex reasoning. Aligned Chain-of-Thought (CoT) models (e.g.,
DeepSeek-distilled models), for instance, leverage this effect; by generating
longer reasoning chains, they push generation into later, more deterministic
(lower BF) stages, resulting in more stable outputs. We hypothesize that
alignment tuning does not fundamentally change a model's behavior, but instead
steers it toward stylistic tokens (e.g., "Sure") that unlock low-entropy
trajectories already present in the base model. This view is supported by
nudging experiments, which show that prompting base models with such tokens can
similarly reduce BF. Together, our findings establish BF as a powerful
diagnostic for understanding and controlling LLM outputs - clarifying how
alignment reduces variability, how CoT promotes stable generations, and how
base models can be steered away from diversity.
DIP: Unsupervised Dense In-Context Post-training of Visual
  Representations
Authors: name: Sophia Sirko-Galouchenko, name: Spyros Gidaris, name: Antonin Vobecky, name: Andrei Bursuc, name: Nicolas Thome
Published: 2025-06-23 10:01:14

We introduce DIP, a novel unsupervised post-training method designed to
enhance dense image representations in large-scale pretrained vision encoders
for in-context scene understanding. Unlike prior approaches that rely on
complex self-distillation architectures, our method trains the vision encoder
using pseudo-tasks that explicitly simulate downstream in-context scenarios,
inspired by meta-learning principles. To enable post-training on unlabeled
data, we propose an automatic mechanism for generating in-context tasks that
combines a pretrained diffusion model and the vision encoder itself. DIP is
simple, unsupervised, and computationally efficient, requiring less than 9
hours on a single A100 GPU. By learning dense representations through pseudo
in-context tasks, it achieves strong performance across a wide variety of
downstream real-world in-context scene understanding tasks. It outperforms both
the initial vision encoder and prior methods, offering a practical and
effective solution for improving dense representations. Code available here:
https://github.com/sirkosophia/DIP
GEMeX-ThinkVG: Towards Thinking with Visual Grounding in Medical VQA via
  Reinforcement Learning
Authors: name: Bo Liu, name: Xiangyu Zhao, name: Along He, name: Yidi Chen, name: Huazhu Fu, name: Xiao-Ming Wu
Published: 2025-06-22 08:09:58

Medical visual question answering aims to support clinical decision-making by
enabling models to answer natural language questions based on medical images.
While recent advances in multi-modal learning have significantly improved
performance, current methods still suffer from limited answer reliability and
poor interpretability, impairing the ability of clinicians and patients to
understand and trust model-generated answers. To address this, this work first
proposes a Thinking with Visual Grounding (ThinkVG) dataset wherein the answer
generation is decomposed into intermediate reasoning steps that explicitly
ground relevant visual regions of the medical image, thereby providing
fine-grained explainability. Furthermore, we introduce a novel verifiable
reward mechanism for reinforcement learning to guide post-training, improving
the alignment between the model's reasoning process and its final answer.
Remarkably, our method achieves comparable performance using only one-eighth of
the training data, demonstrating the efficiency and effectiveness of the
proposal. The dataset is available at
https://huggingface.co/datasets/BoKelvin/GEMeX-ThinkVG.
TPTT: Transforming Pretrained Transformer into Titans
Authors: name: Fabien Furfaro
Published: 2025-06-21 10:06:07

Recent advances in large language models (LLMs) have led to remarkable
progress in natural language processing, but their computational and memory
demands remain a significant challenge, particularly for long-context
inference. We introduce TPTT (Transforming Pretrained Transformer into Titans),
a novel framework for enhancing pretrained Transformer models with efficient
linearized attention mechanisms and advanced memory management. TPTT employs
techniques such as Memory as Gate (MaG) and mixed linearized attention (LiZA).
It is fully compatible with the Hugging Face Transformers library, enabling
seamless adaptation of any causal LLM through parameter-efficient fine-tuning
(LoRA) without full retraining. We show the effectiveness of TPTT on the MMLU
benchmark with models of approximately 1 billion parameters, observing
substantial improvements in both efficiency and accuracy. For instance,
Titans-Llama-3.2-1B achieves a 20% increase in Exact Match (EM) over its
baseline. Statistical analyses and comparisons with recent state-of-the-art
methods confirm the practical scalability and robustness of TPTT. Code is
available at https://github.com/fabienfrfr/tptt . Python package at
https://pypi.org/project/tptt/ .
4Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction
  for 4D Scene Generation
Authors: name: Chaoyang Wang, name: Ashkan Mirzaei, name: Vidit Goel, name: Willi Menapace, name: Aliaksandr Siarohin, name: Avalon Vinella, name: Michael Vasilkovsky, name: Ivan Skorokhodov, name: Vladislav Shakhrai, name: Sergey Korolev, name: Sergey Tulyakov, name: Peter Wonka
Published: 2025-06-18 23:44:59

We propose the first framework capable of computing a 4D spatio-temporal grid
of video frames and 3D Gaussian particles for each time step using a
feed-forward architecture. Our architecture has two main components, a 4D video
model and a 4D reconstruction model. In the first part, we analyze current 4D
video diffusion architectures that perform spatial and temporal attention
either sequentially or in parallel within a two-stream design. We highlight the
limitations of existing approaches and introduce a novel fused architecture
that performs spatial and temporal attention within a single layer. The key to
our method is a sparse attention pattern, where tokens attend to others in the
same frame, at the same timestamp, or from the same viewpoint. In the second
part, we extend existing 3D reconstruction algorithms by introducing a Gaussian
head, a camera token replacement algorithm, and additional dynamic layers and
training. Overall, we establish a new state of the art for 4D generation,
improving both visual quality and reconstruction capability.
CultureMERT: Continual Pre-Training for Cross-Cultural Music
  Representation Learning
Authors: name: Angelos-Nikolaos Kanatas, name: Charilaos Papaioannou, name: Alexandros Potamianos
Published: 2025-06-21 21:16:39

Recent advances in music foundation models have improved audio representation
learning, yet their effectiveness across diverse musical traditions remains
limited. We introduce CultureMERT-95M, a multi-culturally adapted foundation
model developed to enhance cross-cultural music representation learning and
understanding. To achieve this, we propose a two-stage continual pre-training
strategy that integrates learning rate re-warming and re-decaying, enabling
stable adaptation even with limited computational resources. Training on a
650-hour multi-cultural data mix, comprising Greek, Turkish, and Indian music
traditions, results in an average improvement of 4.9% in ROC-AUC and AP across
diverse non-Western music auto-tagging tasks, surpassing prior
state-of-the-art, with minimal forgetting on Western-centric benchmarks. We
further investigate task arithmetic, an alternative approach to multi-cultural
adaptation that merges single-culture adapted models in the weight space. Task
arithmetic performs on par with our multi-culturally trained model on
non-Western auto-tagging tasks and shows no regression on Western datasets.
Cross-cultural evaluation reveals that single-culture models transfer with
varying effectiveness across musical traditions, whereas the multi-culturally
adapted model achieves the best overall performance. To support research on
world music representation learning, we publicly release CultureMERT-95M and
CultureMERT-TA-95M, fostering the development of more culturally aware music
foundation models.
TC-Light: Temporally Consistent Relighting for Dynamic Long Videos
Authors: name: Yang Liu, name: Chuanchen Luo, name: Zimo Tang, name: Yingyan Li, name: Yuran Yang, name: Yuanyong Ning, name: Lue Fan, name: Junran Peng, name: Zhaoxiang Zhang
Published: 2025-06-23 17:59:58

Editing illumination in long videos with complex dynamics has significant
value in various downstream tasks, including visual content creation and
manipulation, as well as data scaling up for embodied AI through sim2real and
real2real transfer. Nevertheless, existing video relighting techniques are
predominantly limited to portrait videos or fall into the bottleneck of
temporal consistency and computation efficiency. In this paper, we propose
TC-Light, a novel paradigm characterized by the proposed two-stage post
optimization mechanism. Starting from the video preliminarily relighted by an
inflated video relighting model, it optimizes appearance embedding in the first
stage to align global illumination. Then it optimizes the proposed canonical
video representation, i.e., Unique Video Tensor (UVT), to align fine-grained
texture and lighting in the second stage. To comprehensively evaluate
performance, we also establish a long and highly dynamic video benchmark.
Extensive experiments show that our method enables physically plausible
relighting results with superior temporal coherence and low computation cost.
The code and video demos are available at
https://dekuliutesla.github.io/tclight/.
Steering Conceptual Bias via Transformer Latent-Subspace Activation
Authors: name: Vansh Sharma, name: Venkat Raman
Published: 2025-06-23 17:56:34

This work examines whether activating latent subspaces in language models
(LLMs) can steer scientific code generation toward a specific programming
language. Five causal LLMs were first evaluated on scientific coding prompts to
quantify their baseline bias among four programming languages. A static
neuron-attribution method, perturbing the highest activated MLP weight for a
C++ or CPP token, proved brittle and exhibited limited generalization across
prompt styles and model scales. To address these limitations, a
gradient-refined adaptive activation steering framework (G-ACT) was developed:
per-prompt activation differences are clustered into a small set of steering
directions, and lightweight per-layer probes are trained and refined online to
select the appropriate steering vector. In LLaMA-3.2 3B, this approach reliably
biases generation towards the CPP language by increasing the average probe
classification accuracy by 15% and the early layers (0-6) improving the probe
classification accuracy by 61.5% compared to the standard ACT framework. For
LLaMA-3.3 70B, where attention-head signals become more diffuse, targeted
injections at key layers still improve language selection. Although per-layer
probing introduces a modest inference overhead, it remains practical by
steering only a subset of layers and enables reproducible model behavior. These
results demonstrate a scalable, interpretable and efficient mechanism for
concept-level control for practical agentic systems.