Orionnnn/hub-stats huggingface dataset:
pLSTM: parallelizable Linear Source Transition Mark networks
Authors: name: Korbinian Pöppel, name: Richard Freinschlag, name: Thomas Schmied, name: Wei Lin, name: Sepp Hochreiter
Published: 2025-06-13 17:51:37

Modern recurrent architectures, such as xLSTM and Mamba, have recently
challenged the Transformer in language modeling. However, their structure
constrains their applicability to sequences only or requires processing
multi-dimensional data structures, such as images or molecular graphs, in a
pre-defined sequential order. In contrast, Multi-Dimensional RNNs (MDRNNs) are
well suited for data with a higher level structure, like 2D grids, trees, and
directed acyclic graphs (DAGs). In this work, we extend the notion of
multi-dimensionality to linear RNNs. We introduce parallelizable Linear Source
Transition Mark networks (pLSTMs) using Source, Transition, and Mark gates that
act on the line graph of a general DAG. This enables parallelization in analogy
to parallel associative scans and the chunkwise-recurrent form of sequential
linear RNNs, but for DAGs. For regular grids (1D and 2D), like images, this
scheme can be efficiently implemented using einsum operations, concatenations,
and padding in logarithmic time. pLSTMs tackle the vanishing/exploding
activation/gradient problem for long distances in DAGs via two distinct modes:
a directed propagation mode (P-mode) and a diffusive distribution mode
(D-mode). To showcase the long-range capabilities of pLSTM, we introduce
arrow-pointing extrapolation as a synthetic computer vision task that contains
long-distance directional information. We demonstrate that pLSTMs generalize
well to larger image sizes, whereas Transformers struggle to extrapolate. On
established molecular graph and computer vision benchmarks, pLSTMs also show
strong performance. Code and Datasets are available at:
https://github.com/ml-jku/plstm_experiments.
Effective Red-Teaming of Policy-Adherent Agents
Authors: name: Itay Nakash, name: George Kour, name: Koren Lazar, name: Matan Vetzler, name: Guy Uziel, name: Ateret Anaby-Tavor
Published: 2025-06-11 10:59:47

Task-oriented LLM-based agents are increasingly used in domains with strict
policies, such as refund eligibility or cancellation rules. The challenge lies
in ensuring that the agent consistently adheres to these rules and policies,
appropriately refusing any request that would violate them, while still
maintaining a helpful and natural interaction. This calls for the development
of tailored design and evaluation methodologies to ensure agent resilience
against malicious user behavior. We propose a novel threat model that focuses
on adversarial users aiming to exploit policy-adherent agents for personal
benefit. To address this, we present CRAFT, a multi-agent red-teaming system
that leverages policy-aware persuasive strategies to undermine a
policy-adherent agent in a customer-service scenario, outperforming
conventional jailbreak methods such as DAN prompts, emotional manipulation, and
coercive. Building upon the existing tau-bench benchmark, we introduce
tau-break, a complementary benchmark designed to rigorously assess the agent's
robustness against manipulative user behavior. Finally, we evaluate several
straightforward yet effective defense strategies. While these measures provide
some protection, they fall short, highlighting the need for stronger,
research-driven safeguards to protect policy-adherent agents from adversarial
attacks
Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity
  Dilemma of Embeddings
Authors: name: Liyan Xu, name: Zhenlin Su, name: Mo Yu, name: Jiangnan Li, name: Fandong Meng, name: Jie Zhou
Published: 2025-06-10 09:00:33

This work focuses on an observed limitation of text encoders: embeddings may
not be able to recognize fine-grained entities or events within the semantics,
resulting in failed dense retrieval on even simple cases. To examine such
behaviors, we first introduce a new evaluation dataset in Chinese, named
CapRetrieval, whose passages are image captions, and queries are phrases
inquiring entities or events in various forms. Zero-shot evaluation suggests
that encoders may fail on these fine-grained matching, regardless of training
sources or model sizes. Aiming for enhancement, we proceed to finetune encoders
with our proposed data generation strategies, which obtains the best
performance on CapRetrieval. Within this process, we further identify an issue
of granularity dilemma, a challenge for embeddings to express fine-grained
salience while aligning with overall semantics. Our dataset, code and models in
this work are publicly released at https://github.com/lxucs/CapRetrieval.
Detecting Harmful Memes with Decoupled Understanding and Guided CoT
  Reasoning
Authors: name: Fengjun Pan, name: Anh Tuan Luu, name: Xiaobao Wu
Published: 2025-06-10 06:10:45

Detecting harmful memes is essential for maintaining the integrity of online
environments. However, current approaches often struggle with resource
efficiency, flexibility, or explainability, limiting their practical deployment
in content moderation systems. To address these challenges, we introduce
U-CoT+, a novel framework for harmful meme detection. Instead of relying solely
on prompting or fine-tuning multimodal models, we first develop a high-fidelity
meme-to-text pipeline that converts visual memes into detail-preserving textual
descriptions. This design decouples meme interpretation from meme
classification, thus avoiding immediate reasoning over complex raw visual
content and enabling resource-efficient harmful meme detection with general
large language models (LLMs). Building on these textual descriptions, we
further incorporate targeted, interpretable human-crafted guidelines to guide
models' reasoning under zero-shot CoT prompting. As such, this framework allows
for easy adaptation to different harmfulness detection criteria across
platforms, regions, and over time, offering high flexibility and
explainability. Extensive experiments on seven benchmark datasets validate the
effectiveness of our framework, highlighting its potential for explainable and
low-resource harmful meme detection using small-scale LLMs. Codes and data are
available at: https://anonymous.4open.science/r/HMC-AF2B/README.md.
The Diffusion Duality
Authors: name: Subham Sekhar Sahoo, name: Justin Deschenaux, name: Aaron Gokaslan, name: Guanghan Wang, name: Justin Chiu, name: Volodymyr Kuleshov
Published: 2025-06-12 16:55:35

Uniform-state discrete diffusion models hold the promise of fast text
generation due to their inherent ability to self-correct. However, they are
typically outperformed by autoregressive models and masked diffusion models. In
this work, we narrow this performance gap by leveraging a key insight:
Uniform-state diffusion processes naturally emerge from an underlying Gaussian
diffusion. Our method, Duo, transfers powerful techniques from Gaussian
diffusion to improve both training and sampling. First, we introduce a
curriculum learning strategy guided by the Gaussian process, doubling training
speed by reducing variance. Models trained with curriculum learning surpass
autoregressive models in zero-shot perplexity on 3 of 7 benchmarks. Second, we
present Discrete Consistency Distillation, which adapts consistency
distillation from the continuous to the discrete setting. This algorithm
unlocks few-step generation in diffusion language models by accelerating
sampling by two orders of magnitude. We provide the code and model checkpoints
on the project page: http://s-sahoo.github.io/duo
SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation
  via Skill Blending
Authors: name: Yuxuan Kuang, name: Haoran Geng, name: Amine Elhafsi, name: Tan-Dzung Do, name: Pieter Abbeel, name: Jitendra Malik, name: Marco Pavone, name: Yue Wang
Published: 2025-06-11 03:24:26

Humanoid robots hold significant potential in accomplishing daily tasks
across diverse environments thanks to their flexibility and human-like
morphology. Recent works have made significant progress in humanoid whole-body
control and loco-manipulation leveraging optimal control or reinforcement
learning. However, these methods require tedious task-specific tuning for each
task to achieve satisfactory behaviors, limiting their versatility and
scalability to diverse tasks in daily scenarios. To that end, we introduce
SkillBlender, a novel hierarchical reinforcement learning framework for
versatile humanoid loco-manipulation. SkillBlender first pretrains
goal-conditioned task-agnostic primitive skills, and then dynamically blends
these skills to accomplish complex loco-manipulation tasks with minimal
task-specific reward engineering. We also introduce SkillBench, a parallel,
cross-embodiment, and diverse simulated benchmark containing three embodiments,
four primitive skills, and eight challenging loco-manipulation tasks,
accompanied by a set of scientific evaluation metrics balancing accuracy and
feasibility. Extensive simulated experiments show that our method significantly
outperforms all baselines, while naturally regularizing behaviors to avoid
reward hacking, resulting in more accurate and feasible movements for diverse
loco-manipulation tasks in our daily scenarios. Our code and benchmark will be
open-sourced to the community to facilitate future research. Project page:
https://usc-gvl.github.io/SkillBlender-web/.
Aligned Novel View Image and Geometry Synthesis via Cross-modal
  Attention Instillation
Authors: name: Min-Seop Kwak, name: Junho Kim, name: Sangdoo Yun, name: Dongyoon Han, name: Taekyoung Kim, name: Seungryong Kim, name: Jin-Hwa Kim
Published: 2025-06-13 16:19:00

We introduce a diffusion-based framework that performs aligned novel view
image and geometry generation via a warping-and-inpainting methodology. Unlike
prior methods that require dense posed images or pose-embedded generative
models limited to in-domain views, our method leverages off-the-shelf geometry
predictors to predict partial geometries viewed from reference images, and
formulates novel-view synthesis as an inpainting task for both image and
geometry. To ensure accurate alignment between generated images and geometry,
we propose cross-modal attention distillation, where attention maps from the
image diffusion branch are injected into a parallel geometry diffusion branch
during both training and inference. This multi-task approach achieves
synergistic effects, facilitating geometrically robust image synthesis as well
as well-defined geometry prediction. We further introduce proximity-based mesh
conditioning to integrate depth and normal cues, interpolating between point
cloud and filtering erroneously predicted geometry from influencing the
generation process. Empirically, our method achieves high-fidelity
extrapolative view synthesis on both image and geometry across a range of
unseen scenes, delivers competitive reconstruction quality under interpolation
settings, and produces geometrically aligned colored point clouds for
comprehensive 3D completion. Project page is available at
https://cvlab-kaist.github.io/MoAI.
LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive
  Programming?
Authors: name: Zihan Zheng, name: Zerui Cheng, name: Zeyu Shen, name: Shang Zhou, name: Kaiyuan Liu, name: Hansen He, name: Dongruixuan Li, name: Stanley Wei, name: Hangyi Hao, name: Jianzhu Yao, name: Peiyao Sheng, name: Zixuan Wang, name: Wenhao Chai, name: Aleksandra Korolova, name: Peter Henderson, name: Sanjeev Arora, name: Pramod Viswanath, name: Jingbo Shang, name: Saining Xie
Published: 2025-06-13 16:29:09

Recent reports claim that large language models (LLMs) now outperform elite
humans in competitive programming. Drawing on knowledge from a group of
medalists in international algorithmic contests, we revisit this claim,
examining how LLMs differ from human experts and where limitations still
remain. We introduce LiveCodeBench Pro, a benchmark composed of problems from
Codeforces, ICPC, and IOI that are continuously updated to reduce the
likelihood of data contamination. A team of Olympiad medalists annotates every
problem for algorithmic categories and conducts a line-by-line analysis of
failed model-generated submissions. Using this new data and benchmark, we find
that frontier models still have significant limitations: without external
tools, the best model achieves only 53% pass@1 on medium-difficulty problems
and 0% on hard problems, domains where expert humans still excel. We also find
that LLMs succeed at implementation-heavy problems but struggle with nuanced
algorithmic reasoning and complex case analysis, often generating confidently
incorrect justifications. High performance appears largely driven by
implementation precision and tool augmentation, not superior reasoning.
LiveCodeBench Pro thus highlights the significant gap to human grandmaster
levels, while offering fine-grained diagnostics to steer future improvements in
code-centric LLM reasoning.
Configurable Preference Tuning with Rubric-Guided Synthetic Data
Authors: name: Víctor Gallego
Published: 2025-06-13 12:17:38

Models of human feedback for AI alignment, such as those underpinning Direct
Preference Optimization (DPO), often bake in a singular, static set of
preferences, limiting adaptability. This paper challenges the assumption of
monolithic preferences by introducing Configurable Preference Tuning (CPT), a
novel framework for endowing language models with the ability to dynamically
adjust their behavior based on explicit, human-interpretable directives. CPT
leverages synthetically generated preference data, conditioned on system
prompts derived from structured, fine-grained rubrics that define desired
attributes like writing style. By fine-tuning with these rubric-guided
preferences, the LLM learns to modulate its outputs at inference time in
response to the system prompt, without retraining. This approach not only
offers fine-grained control but also provides a mechanism for modeling more
nuanced and context-dependent human feedback. Several experimental artifacts,
such as training code, generated datasets and fine-tuned models are released at
https://github.com/vicgalle/configurable-preference-tuning
A High-Quality Dataset and Reliable Evaluation for Interleaved
  Image-Text Generation
Authors: name: Yukang Feng, name: Jianwen Sun, name: Chuanhao Li, name: Zizhen Li, name: Jiaxin Ai, name: Fanrui Zhang, name: Yifan Chang, name: Sizhuo Zhou, name: Shenglin Zhang, name: Yu Dai, name: Kaipeng Zhang
Published: 2025-06-11 06:21:20

Recent advancements in Large Multimodal Models (LMMs) have significantly
improved multimodal understanding and generation. However, these models still
struggle to generate tightly interleaved image-text outputs, primarily due to
the limited scale, quality and instructional richness of current training
datasets. To address this, we introduce InterSyn, a large-scale multimodal
dataset constructed using our Self-Evaluation with Iterative Refinement (SEIR)
method. InterSyn features multi-turn, instruction-driven dialogues with tightly
interleaved imagetext responses, providing rich object diversity and rigorous
automated quality refinement, making it well-suited for training
next-generation instruction-following LMMs. Furthermore, to address the lack of
reliable evaluation tools capable of assessing interleaved multimodal outputs,
we introduce SynJudge, an automatic evaluation model designed to quantitatively
assess multimodal outputs along four dimensions: text content, image content,
image quality, and image-text synergy.
  Experimental studies show that the SEIR method leads to substantially higher
dataset quality compared to an otherwise identical process without refinement.
  Moreover, LMMs trained on InterSyn achieve uniform performance gains across
all evaluation metrics, confirming InterSyn's utility for advancing multimodal
systems.
Comment on The Illusion of Thinking: Understanding the Strengths and
  Limitations of Reasoning Models via the Lens of Problem Complexity
Authors: name: C. Opus, name: A. Lawsen
Published: 2025-06-10 21:16:53

Shojaee et al. (2025) report that Large Reasoning Models (LRMs) exhibit
"accuracy collapse" on planning puzzles beyond certain complexity thresholds.
We demonstrate that their findings primarily reflect experimental design
limitations rather than fundamental reasoning failures. Our analysis reveals
three critical issues: (1) Tower of Hanoi experiments systematically exceed
model output token limits at reported failure points, with models explicitly
acknowledging these constraints in their outputs; (2) The authors' automated
evaluation framework fails to distinguish between reasoning failures and
practical constraints, leading to misclassification of model capabilities; (3)
Most concerningly, their River Crossing benchmarks include mathematically
impossible instances for N > 5 due to insufficient boat capacity, yet models
are scored as failures for not solving these unsolvable problems. When we
control for these experimental artifacts, by requesting generating functions
instead of exhaustive move lists, preliminary experiments across multiple
models indicate high accuracy on Tower of Hanoi instances previously reported
as complete failures. These findings highlight the importance of careful
experimental design when evaluating AI reasoning capabilities.
NoLoCo: No-all-reduce Low Communication Training Method for Large Models
Authors: name: Jari Kolehmainen, name: Nikolay Blagoev, name: John Donaghy, name: Oğuzhan Ersoy, name: Christopher Nies
Published: 2025-06-12 17:23:23

Training large language models is generally done via optimization methods on
clusters containing tens of thousands of accelerators, communicating over a
high-bandwidth interconnect. Scaling up these clusters is expensive and can
become impractical, imposing limits on the size of models that can be trained.
Several recent studies have proposed training methods that are less
communication intensive, avoiding the need for a highly connected compute
cluster. These state-of-the-art low communication training methods still employ
a synchronization step for model parameters, which, when performed over all
model replicas, can become costly on a low-bandwidth network.
  In this work, we propose a novel optimization method, NoLoCo, that does not
explicitly synchronize all model parameters during training and, as a result,
does not require any collective communication. NoLoCo implicitly synchronizes
model weights via a novel variant of the Nesterov momentum optimizer by
partially averaging model weights with a randomly selected other one. We
provide both a theoretical convergence analysis for our proposed optimizer as
well as empirical results from language model training.
  We benchmark NoLoCo on a wide range of accelerator counts and model sizes,
between 125M to 6.8B parameters. Our method requires significantly less
communication overhead than fully sharded data parallel training or even widely
used low communication training method, DiLoCo. The synchronization step itself
is estimated to be one magnitude faster than the all-reduce used in DiLoCo for
few hundred accelerators training over the internet. We also do not have any
global blocking communication that reduces accelerator idling time. Compared to
DiLoCo, we also observe up to 4% faster convergence rate with wide range of
model sizes and accelerator counts.
LaTtE-Flow: Layerwise Timestep-Expert Flow-based Transformer
Authors: name: Ying Shen, name: Zhiyang Xu, name: Jiuhai Chen, name: Shizhe Diao, name: Jiaxin Zhang, name: Yuguang Yao, name: Joy Rimchala, name: Ismini Lourentzou, name: Lifu Huang
Published: 2025-06-08 00:15:32

Recent advances in multimodal foundation models unifying image understanding
and generation have opened exciting avenues for tackling a wide range of
vision-language tasks within a single framework. Despite progress, existing
unified models typically require extensive pretraining and struggle to achieve
the same level of performance compared to models dedicated to each task.
Additionally, many of these models suffer from slow image generation speeds,
limiting their practical deployment in real-time or resource-constrained
settings. In this work, we propose Layerwise Timestep-Expert Flow-based
Transformer (LaTtE-Flow), a novel and efficient architecture that unifies image
understanding and generation within a single multimodal model. LaTtE-Flow
builds upon powerful pretrained Vision-Language Models (VLMs) to inherit strong
multimodal understanding capabilities, and extends them with a novel Layerwise
Timestep Experts flow-based architecture for efficient image generation.
LaTtE-Flow distributes the flow-matching process across specialized groups of
Transformer layers, each responsible for a distinct subset of timesteps. This
design significantly improves sampling efficiency by activating only a small
subset of layers at each sampling timestep. To further enhance performance, we
propose a Timestep-Conditioned Residual Attention mechanism for efficient
information reuse across layers. Experiments demonstrate that LaTtE-Flow
achieves strong performance on multimodal understanding tasks, while achieving
competitive image generation quality with around 6x faster inference speed
compared to recent unified multimodal models.
Decomposing MLP Activations into Interpretable Features via
  Semi-Nonnegative Matrix Factorization
Authors: name: Or Shafran, name: Atticus Geiger, name: Mor Geva
Published: 2025-06-12 17:33:29

A central goal for mechanistic interpretability has been to identify the
right units of analysis in large language models (LLMs) that causally explain
their outputs. While early work focused on individual neurons, evidence that
neurons often encode multiple concepts has motivated a shift toward analyzing
directions in activation space. A key question is how to find directions that
capture interpretable features in an unsupervised manner. Current methods rely
on dictionary learning with sparse autoencoders (SAEs), commonly trained over
residual stream activations to learn directions from scratch. However, SAEs
often struggle in causal evaluations and lack intrinsic interpretability, as
their learning is not explicitly tied to the computations of the model. Here,
we tackle these limitations by directly decomposing MLP activations with
semi-nonnegative matrix factorization (SNMF), such that the learned features
are (a) sparse linear combinations of co-activated neurons, and (b) mapped to
their activating inputs, making them directly interpretable. Experiments on
Llama 3.1, Gemma 2 and GPT-2 show that SNMF derived features outperform SAEs
and a strong supervised baseline (difference-in-means) on causal steering,
while aligning with human-interpretable concepts. Further analysis reveals that
specific neuron combinations are reused across semantically-related features,
exposing a hierarchical structure in the MLP's activation space. Together,
these results position SNMF as a simple and effective tool for identifying
interpretable features and dissecting concept representations in LLMs.