Orionnnn/hub-stats huggingface dataset:
pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation
Authors: name: Hansheng Chen, name: Kai Zhang, name: Hao Tan, name: Leonidas Guibas, name: Gordon Wetzstein, name: Sai Bi
Published: 2025-10-16 17:59:51

Few-step diffusion or flow-based generative models typically distill a
velocity-predicting teacher into a student that predicts a shortcut towards
denoised data. This format mismatch has led to complex distillation procedures
that often suffer from a quality-diversity trade-off. To address this, we
propose policy-based flow models (pi-Flow). pi-Flow modifies the output
layer of a student flow model to predict a network-free policy at one timestep.
The policy then produces dynamic flow velocities at future substeps with
negligible overhead, enabling fast and accurate ODE integration on these
substeps without extra network evaluations. To match the policy's ODE
trajectory to the teacher's, we introduce a novel imitation distillation
approach, which matches the policy's velocity to the teacher's along the
policy's trajectory using a standard ell_2 flow matching loss. By simply
mimicking the teacher's behavior, pi-Flow enables stable and scalable
training and avoids the quality-diversity trade-off. On ImageNet 256^2, it
attains a 1-NFE FID of 2.85, outperforming MeanFlow of the same DiT
architecture. On FLUX.1-12B and Qwen-Image-20B at 4 NFEs, pi-Flow achieves
substantially better diversity than state-of-the-art few-step methods, while
maintaining teacher-level quality.
LLM-guided Hierarchical Retrieval
Authors: name: Nilesh Gupta, name: Wei-Cheng Chang, name: Ngot Bui, name: Cho-Jui Hsieh, name: Inderjit S. Dhillon
Published: 2025-10-15 07:05:17

Modern IR systems are increasingly tasked with answering complex,
multi-faceted queries that require deep reasoning rather than simple keyword or
semantic matching. While LLM-based IR has shown great promise, the prevailing
retrieve-then-rerank paradigm inherits the limitations of embedding-based
retrieval; parametric generative approaches are difficult to update with new
information; and long-context methods that place the entire corpus in context
are computationally infeasible for large document collections. To address these
challenges, we introduce LATTICE, a hierarchical retrieval framework that
enables an LLM to reason over and navigate large corpora with logarithmic
search complexity by imposing a semantic tree structure on the corpus. Our
approach consists of two stages: (1) an offline phase that organizes the corpus
into a semantic hierarchy via either a bottom-up agglomerative strategy or a
top-down divisive strategy using multi-level summaries and (2) an online
traversal phase where a search LLM navigates this tree. A central challenge in
such LLM-guided search is that the model's relevance judgments are noisy,
context-dependent, and unaware of the hierarchy, making cross-branch and
cross-level comparisons difficult. To overcome this, we propose a traversal
algorithm that estimates calibrated latent relevance scores from local LLM
outputs and aggregates them into a global path relevance metric. Our
training-free framework achieves state-of-the-art zero-shot performance on the
reasoning-intensive BRIGHT benchmark, demonstrating up to 9% improvement in
Recall@100 and 5% in nDCG@10 over the next best zero-shot baseline.
Furthermore, compared to the fine-tuned SOTA method DIVER-v2, LATTICE attains
comparable results on BRIGHT subsets that use a static corpus for evaluation.
VIST3A: Text-to-3D by Stitching a Multi-view Reconstruction Network to a
  Video Generator
Authors: name: Hyojun Go, name: Dominik Narnhofer, name: Goutam Bhat, name: Prune Truong, name: Federico Tombari, name: Konrad Schindler
Published: 2025-10-15 11:55:08

The rapid progress of large, pretrained models for both visual content
generation and 3D reconstruction opens up new possibilities for text-to-3D
generation. Intuitively, one could obtain a formidable 3D scene generator if
one were able to combine the power of a modern latent text-to-video model as
"generator" with the geometric abilities of a recent (feedforward) 3D
reconstruction system as "decoder". We introduce VIST3A, a general framework
that does just that, addressing two main challenges. First, the two components
must be joined in a way that preserves the rich knowledge encoded in their
weights. We revisit model stitching, i.e., we identify the layer in the 3D
decoder that best matches the latent representation produced by the
text-to-video generator and stitch the two parts together. That operation
requires only a small dataset and no labels. Second, the text-to-video
generator must be aligned with the stitched 3D decoder, to ensure that the
generated latents are decodable into consistent, perceptually convincing 3D
scene geometry. To that end, we adapt direct reward finetuning, a popular
technique for human preference alignment. We evaluate the proposed VIST3A
approach with different video generators and 3D reconstruction models. All
tested pairings markedly improve over prior text-to-3D models that output
Gaussian splats. Moreover, by choosing a suitable 3D base model, VIST3A also
enables high-quality text-to-pointmap generation.
Beyond One World: Benchmarking Super Heros in Role-Playing Across
  Multiversal Contexts
Authors: name: Perapard Ngokpol, name: Kun Kerdthaisong, name: Pasin Buakhaw, name: Pitikorn Khlaisamniang, name: Supasate Vorathammathorn, name: Piyalitt Ittichaiwong, name: Nutchanon Yongsatianchot
Published: 2025-10-16 06:39:27

Large language models (LLMs) are increasingly used as role-playing agents,
yet their capacity to faithfully and consistently portray version-specific
characters -- for example, superheroes across comic and cinematic universes --
remains underexplored. Superhero canons such as Marvel and DC provide a rich
testbed: decades of storytelling yield multiple incarnations of the same
character with distinct histories, values, and moral codes. To study this
problem, we introduce Beyond One World, a benchmark for character-grounded
roleplay spanning 30 iconic heroes and 90 canon-specific versions. The
benchmark comprises two tasks: (i) Canon Events, which probes factual recall of
pivotal life stages, and (ii) Moral Dilemmas, which confronts models with
ethically charged scenarios. We score responses for canonical accuracy and
reasoning fidelity under a framework that separates internal deliberation
("thinking") from outward decisions ("acting"). We further propose Think-Act
Matching, a metric that quantifies alignment between reasons and actions and
serves as a proxy for model trustworthiness. Experiments across reasoning- and
non-reasoning-oriented models yield three findings: (1) chain-of-thought
prompting improves narrative coherence in weaker models but can reduce
canonical accuracy in stronger ones; (2) cross-version generalization within a
character remains a major obstacle; and (3) models often excel at either
thinking or acting, but rarely both. Beyond One World exposes critical gaps in
multiversal consistency and reasoning alignment, offering a challenging
evaluation for role-playing LLMs.
KVCOMM: Online Cross-context KV-cache Communication for Efficient
  LLM-based Multi-agent Systems
Authors: name: Hancheng Ye, name: Zhengqi Gao, name: Mingyuan Ma, name: Qinsi Wang, name: Yuzhe Fu, name: Ming-Yu Chung, name: Yueqian Lin, name: Zhijian Liu, name: Jianyi Zhang, name: Danyang Zhuo, name: Yiran Chen
Published: 2025-10-14 18:00:01

Multi-agent large language model (LLM) systems are increasingly adopted for
complex language processing tasks that require communication and coordination
among agents. However, these systems often suffer substantial overhead from
repeated reprocessing of overlapping contexts across agents. In typical
pipelines, once an agent receives a message from its predecessor, the full
context-including prior turns-must be reprocessed from scratch, leading to
inefficient processing. While key-value (KV) caching is an effective solution
for avoiding redundant computation in single-agent settings where prefixes
remain unchanged, it cannot be directly reused in multi-agent scenarios due to
diverging prefixes introduced by agent-specific context extensions. We identify
that the core challenge lies in the offset variance of KV-caches across agents.
To address this, we propose KVCOMM, a training-free framework that enables
efficient prefilling in multi-agent inference by reusing KV-caches and aligning
cache offsets of overlapping contexts under diverse prefix contexts. KVCOMM
estimates and adjusts KV-caches for shared content by referencing a pool of
cached examples-termed anchors-that store observed cache deviations under
varying prefixes. The anchor pool is maintained and updated online, allowing
dynamic adaptation to distinct user requests and context structures. KVCOMM
achieves over 70% reuse rate across diverse multi-agent workloads, including
retrieval-augmented generation, math reasoning, and collaborative coding tasks,
all without quality degradation. Particularly, when each fully-connected agent
receives 1K input tokens with 512 prefix tokens and 512 output tokens under a
five-agent setting, KVCOMM achieves up to 7.8x speedup compared to the standard
prefill pipeline, reducing TTFT from ~430 ms to ~55 ms.
Less is More: Improving LLM Reasoning with Minimal Test-Time
  Intervention
Authors: name: Zhen Yang, name: Mingyang Zhang, name: Feng Chen, name: Ganggui Ding, name: Liang Hou, name: Xin Tao, name: Pengfei Wan, name: Ying-Cong Chen
Published: 2025-10-15 17:59:45

Recent progress in large language models (LLMs) has focused on test-time
scaling to improve reasoning via increased inference computation, but often at
the cost of efficiency. We revisit test-time behavior and uncover a simple yet
underexplored phenomenon: reasoning uncertainty is highly localized-only a
small subset of high-entropy tokens dominantly affects output correctness.
Motivated by this, we propose Minimal Test-Time Intervention (MTI), a
training-free framework that enhances reasoning accuracy and stability with
minimal overhead. MTI includes: (i) Selective CFG intervention, applying
classifier-free guidance only at uncertain positions; and (ii) Lightweight
negative-prompt guidance, reusing the main model's KV cache to approximate
unconditional decoding efficiently. MTI yields consistent gains across general,
coding, and STEM tasks-e.g., +1.35% average improvement on eight benchmarks for
Qwen3-8B-Base and +5% on AIME2024 using Qwen3-32B-Reasoning-while remaining
highly efficient.
Don't Throw Away Your Pretrained Model
Authors: name: Shangbin Feng, name: Wenhao Yu, name: Yike Wang, name: Hongming Zhang, name: Yulia Tsvetkov, name: Dong Yu
Published: 2025-10-10 23:12:20

Alignment training has tradeoffs: it helps language models (LMs) gain in
reasoning and instruction following but might lose out on skills such as
creativity and calibration, where unaligned base models are better at. We aim
to make the best of both worlds through model collaboration, where different
models in the training pipeline collaborate and complement each other. Since LM
responses feature interleaving skills that favor different models, we propose
Switch Generation, where pretrained and aligned model versions take turns to
``speak'' in a response sequence. Specifically, we train a switcher LM by
learning from outcomes of choosing different models to generate the next
segment across diverse queries and contexts. At inference time, the switcher LM
guides different model checkpoints to dynamically generate the next segment
where their strengths are most needed. Extensive experiments with 8 model
collaboration baselines and 18 datasets show that 1) model collaboration
consistently outperforms individual models on 16 out of 18 tasks, and 2) Switch
Generation further outperforms baselines by 12.9% on average. Further analysis
reveals that Switch Generation discovers compositional skills to solve problems
where individual models struggle and generalizes to unseen models and tasks,
reusing and repurposing by-products in expensive model training pipelines that
are otherwise discarded.
Learning to Grasp Anything by Playing with Random Toys
Authors: name: Dantong Niu, name: Yuvan Sharma, name: Baifeng Shi, name: Rachel Ding, name: Matteo Gioia, name: Haoru Xue, name: Henry Tsai, name: Konstantinos Kallidromitis, name: Anirudh Pai, name: Shankar Shastry, name: Trevor Darrell, name: Jitendra Malik, name: Roei Herzig
Published: 2025-10-14 17:56:10

Robotic manipulation policies often struggle to generalize to novel objects,
limiting their real-world utility. In contrast, cognitive science suggests that
children develop generalizable dexterous manipulation skills by mastering a
small set of simple toys and then applying that knowledge to more complex
items. Inspired by this, we study if similar generalization capabilities can
also be achieved by robots. Our results indicate robots can learn generalizable
grasping using randomly assembled objects that are composed from just four
shape primitives: spheres, cuboids, cylinders, and rings. We show that training
on these "toys" enables robust generalization to real-world objects, yielding
strong zero-shot performance. Crucially, we find the key to this generalization
is an object-centric visual representation induced by our proposed detection
pooling mechanism. Evaluated in both simulation and on physical robots, our
model achieves a 67% real-world grasping success rate on the YCB dataset,
outperforming state-of-the-art approaches that rely on substantially more
in-domain data. We further study how zero-shot generalization performance
scales by varying the number and diversity of training toys and the
demonstrations per toy. We believe this work offers a promising path to
scalable and generalizable learning in robotic manipulation. Demonstration
videos, code, checkpoints and our dataset are available on our project page:
https://lego-grasp.github.io/ .
Tracing the Traces: Latent Temporal Signals for Efficient and Accurate
  Reasoning
Authors: name: Martina G. Vilas, name: Safoora Yousefi, name: Besmira Nushi, name: Eric Horvitz, name: Vidhisha Balachandran
Published: 2025-10-12 08:03:56

Reasoning models improve their problem-solving ability through inference-time
scaling, allocating more compute via longer token budgets. Identifying which
reasoning traces are likely to succeed remains a key opportunity: reliably
predicting productive paths can substantially reduce wasted computation and
improve overall efficiency. We introduce Latent-Trajectory signals that
characterize the temporal evolution of a model's internal representations
during the generation of intermediate reasoning tokens. By measuring the
overall change in latent representations between the start and end of
reasoning, the change accumulated across intermediate steps, and the extent to
which these changes advance toward the final state, we show that these signals
predict solution accuracy more reliably than both cross-layer metrics and
output-based confidence measures. When used to guide answer selection across
multiple sampled generations, Latent-Trajectory signals make test-time scaling
more effective and efficient than majority voting, reducing token usage by up
to 70% while preserving and even improving accuracy by 2.6% on average.
Moreover, these predictive signals often emerge early in the reasoning trace,
enabling early selection and allocation of compute to the most promising
candidates. Our findings contribute not only practical strategies for
inference-time efficiency, but also a deeper interpretability perspective on
how reasoning processes are represented and differentiated in latent space.
Haystack Engineering: Context Engineering for Heterogeneous and Agentic
  Long-Context Evaluation
Authors: name: Mufei Li, name: Dongqi Fu, name: Limei Wang, name: Si Zhang, name: Hanqing Zeng, name: Kaan Sancak, name: Ruizhong Qiu, name: Haoyu Wang, name: Xiaoxin He, name: Xavier Bresson, name: Yinglong Xia, name: Chonglin Sun, name: Pan Li
Published: 2025-10-08 18:12:10

Modern long-context large language models (LLMs) perform well on synthetic
"needle-in-a-haystack" (NIAH) benchmarks, but such tests overlook how noisy
contexts arise from biased retrieval and agentic workflows. We argue that
haystack engineering is necessary to construct noisy long contexts that
faithfully capture key real-world factors -- distraction from heterogeneous
biased retrievers and cascading errors in agentic workflows -- to test models'
long-context robustness. We instantiate it through HaystackCraft, a new NIAH
benchmark built on the full English Wikipedia hyperlink network with multi-hop
questions. HaystackCraft evaluates how heterogeneous retrieval strategies
(e.g., sparse, dense, hybrid, and graph-based) affect distractor composition,
haystack ordering, and downstream LLM performance. HaystackCraft further
extends NIAH to dynamic, LLM-dependent settings that simulate agentic
operations, where models refine queries, reflect on their past reasonings, and
decide when to stop. Experiments with 15 long-context models show that (1)
while stronger dense retrievers can introduce more challenging distractors,
graph-based reranking simultaneously improves retrieval effectiveness and
mitigates more harmful distractors; (2) in agentic tests, even advanced models
like Gemini 2.5 Pro and GPT-5 suffer cascading failures from self-generated
distractors or struggle to perform early stops. These results highlight
persistent challenges in agentic long-context reasoning and establish
HaystackCraft as a valuable testbed for future progress.
What Generative Search Engines Like and How to Optimize Web Content
  Cooperatively
Authors: name: Yujiang Wu, name: Shanshan Zhong, name: Yubin Kim, name: Chenyan Xiong
Published: 2025-10-13 14:10:26

By employing large language models (LLMs) to retrieve documents and generate
natural language responses, Generative Engines, such as Google AI overview and
ChatGPT, provide significantly enhanced user experiences and have rapidly
become the new form of search. Their rapid adoption also drives the needs of
Generative Engine Optimization (GEO), as content providers are eager to gain
more traction from them. In this paper, we introduce AutoGEO, a framework to
automatically learn generative engine preferences when using retrieved contents
for response generation, and rewrite web contents for more such traction.
AutoGEO first prompts frontier LLMs to explain generative engine preferences
and extract meaningful preference rules from these explanations. Then it uses
preference rules as context engineering for AutoGEO_API, a
prompt-based GEO system, and as rule-based rewards to train
AutoGEO_Mini, a cost-effective GEO model. Experiments on the standard
GEO-Bench and two newly constructed benchmarks using real user queries
demonstrate the effectiveness of AutoGEO in enhancing content traction while
preserving search utility. Analyses confirm the learned rules' robustness and
abilities to capture unique preferences in variant domains, and AutoGEO
systems' ability to embed them in content optimization. The code is released at
https://github.com/cxcscmu/AutoGEO.
Evaluating Language Models' Evaluations of Games
Authors: name: Katherine M. Collins, name: Cedegao E. Zhang, name: Graham Todd, name: Lance Ying, name: Mauricio Barba da Costa, name: Ryan Liu, name: Prafull Sharma, name: Adrian Weller, name: Ionatan Kuperwajs, name: Lionel Wong, name: Joshua B. Tenenbaum, name: Thomas L. Griffiths
Published: 2025-10-13 02:45:37

Reasoning is not just about solving problems -- it is also about evaluating
which problems are worth solving at all. Evaluations of artificial intelligence
(AI) systems primarily focused on problem solving, historically by studying how
models play games such as chess and Go. In this paper, we advocate for a new
paradigm that assesses AI systems' evaluation of games. First, we introduce a
formalism for evaluating such evaluations. We then leverage a large-scale
dataset of over 100 novel board games and over 450 human judgments to compare
evaluations produced by modern language and reasoning models against those of
people and symbolic computational agents. We consider two kinds of evaluative
queries: assessing the payoff (or fairness) and the funness of games. These
queries span two dimensions relevant to the design of evaluations of AI
evaluations: how complex a query is to compute and how difficult a query is to
quantify. Our results show that reasoning models are generally more aligned to
people in their evaluations of games than non-reasoning language models.
However, we observe a non-monotonic relationship: as models get closer to
game-theoretic optimal, their fit to human data weakens. We also observe more
"jaggedness" across models for assessing funness, in line with the greater
difficulty of quantifying this query. Across queries and games, reasoning
models show highly variable and unpredictable resource usage when assessing
queries, pointing to the importance of imbuing more resource-rational
meta-reasoning in language and reasoning models.
Dedelayed: Deleting remote inference delay via on-device correction
Authors: name: Dan Jacobellis, name: Mateen Ulhaq, name: Fabien Racapé, name: Hyomin Choi, name: Neeraja J. Yadwadkar
Published: 2025-10-15 16:13:44

Remote inference allows lightweight devices to leverage powerful cloud
models. However, communication network latency makes predictions stale and
unsuitable for real-time tasks. To address this, we introduce Dedelayed, a
delay-corrective method that mitigates arbitrary remote inference delays,
allowing the local device to produce low-latency outputs in real time. Our
method employs a lightweight local model that processes the current frame and
fuses in features that a heavyweight remote model computes from past frames. On
video from the BDD100K driving dataset, Dedelayed improves semantic
segmentation accuracy over the stronger of the local-only and remote-only
baselines across all realistic communication network delays beyond 33 ms.
Without incurring additional delay, it improves accuracy by 6.4 mIoU compared
to fully local inference and 9.8 mIoU compared to remote inference, for a
round-trip delay of 100 ms. The advantage grows under longer delays and
higher-motion scenes, as delay-mitigated split inference sustains accuracy more
effectively, providing clear advantages for real-time tasks that must remain
aligned with the current world state.
The Art of Scaling Reinforcement Learning Compute for LLMs
Authors: name: Devvrit Khatri, name: Lovish Madaan, name: Rishabh Tiwari, name: Rachit Bansal, name: Sai Surya Duvvuri, name: Manzil Zaheer, name: Inderjit S. Dhillon, name: David Brandfonbrener, name: Rishabh Agarwal
Published: 2025-10-15 17:43:03

Reinforcement learning (RL) has become central to training large language
models (LLMs), yet the field lacks predictive scaling methodologies comparable
to those established for pre-training. Despite rapidly rising compute budgets,
there is no principled understanding of how to evaluate algorithmic
improvements for scaling RL compute. We present the first large-scale
systematic study, amounting to more than 400,000 GPU-hours, that defines a
principled framework for analyzing and predicting RL scaling in LLMs. We fit
sigmoidal compute-performance curves for RL training and ablate a wide range of
common design choices to analyze their effects on asymptotic performance and
compute efficiency. We observe: (1) Not all recipes yield similar asymptotic
performance, (2) Details such as loss aggregation, normalization, curriculum,
and off-policy algorithm primarily modulate compute efficiency without
materially shifting the asymptote, and (3) Stable, scalable recipes follow
predictable scaling trajectories, enabling extrapolation from smaller-scale
runs. Combining these insights, we propose a best-practice recipe, ScaleRL, and
demonstrate its effectiveness by successfully scaling and predicting validation
performance on a single RL run scaled up to 100,000 GPU-hours. Our work
provides both a scientific framework for analyzing scaling in RL and a
practical recipe that brings RL training closer to the predictability long
achieved in pre-training.