Orionnnn/hub-stats huggingface dataset:
PaSa: An LLM Agent for Comprehensive Academic Paper Search
Authors: name: Yichen He, name: Guanhua Huang, name: Peiyuan Feng, name: Yuan Lin, name: Yuchen Zhang, name: Hang Li, name: Weinan E
Published: 2025-01-17 11:12:28

We introduce PaSa, an advanced Paper Search agent powered by large language
models. PaSa can autonomously make a series of decisions, including invoking
search tools, reading papers, and selecting relevant references, to ultimately
obtain comprehensive and accurate results for complex scholarly queries. We
optimize PaSa using reinforcement learning with a synthetic dataset,
AutoScholarQuery, which includes 35k fine-grained academic queries and
corresponding papers sourced from top-tier AI conference publications.
Additionally, we develop RealScholarQuery, a benchmark collecting real-world
academic queries to assess PaSa performance in more realistic scenarios.
Despite being trained on synthetic data, PaSa significantly outperforms
existing baselines on RealScholarQuery, including Google, Google Scholar,
Google with GPT-4 for paraphrased queries, chatGPT (search-enabled GPT-4o),
GPT-o1, and PaSa-GPT-4o (PaSa implemented by prompting GPT-4o). Notably,
PaSa-7B surpasses the best Google-based baseline, Google with GPT-4o, by 37.78%
in recall@20 and 39.90% in recall@50. It also exceeds PaSa-GPT-4o by 30.36% in
recall and 4.25% in precision. Model, datasets, and code are available at
https://github.com/bytedance/pasa.
ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling
  under Long-Context Scenario
Authors: name: Lucen Zhong, name: Zhengxiao Du, name: Xiaohan Zhang, name: Haiyi Hu, name: Jie Tang
Published: 2025-01-17 11:41:53

Enhancing large language models (LLMs) with real-time APIs can help generate
more accurate and up-to-date responses. However, evaluating the function
calling abilities of LLMs in real-world scenarios remains under-explored due to
the complexity of data collection and evaluation. In this work, we introduce
ComplexFuncBench, a benchmark for complex function calling across five
real-world scenarios. Compared to existing benchmarks, ComplexFuncBench
encompasses multi-step and constrained function calling, which requires
long-parameter filing, parameter value reasoning, and 128k long context.
Additionally, we propose an automatic framework, ComplexEval, for
quantitatively evaluating complex function calling tasks. Through comprehensive
experiments, we demonstrate the deficiencies of state-of-the-art LLMs in
function calling and suggest future directions for optimizing these
capabilities. The data and code are available at
https://github.com/THUDM/ComplexFuncBench.
The Heap: A Contamination-Free Multilingual Code Dataset for Evaluating
  Large Language Models
Authors: name: Jonathan Katzy, name: Razvan Mihai Popescu, name: Arie van Deursen, name: Maliheh Izadi
Published: 2025-01-16 16:48:41

The recent rise in the popularity of large language models has spurred the
development of extensive code datasets needed to train them. This has left
limited code available for collection and use in the downstream investigation
of specific behaviors, or evaluation of large language models without suffering
from data contamination. To address this problem, we release The Heap, a large
multilingual dataset covering 57 programming languages that has been
deduplicated with respect to other open datasets of code, enabling researchers
to conduct fair evaluations of large language models without significant data
cleaning overhead.
Do generative video models learn physical principles from watching
  videos?
Authors: name: Saman Motamed, name: Laura Culp, name: Kevin Swersky, name: Priyank Jaini, name: Robert Geirhos
Published: 2025-01-14 20:59:37

AI video generation is undergoing a revolution, with quality and realism
advancing rapidly. These advances have led to a passionate scientific debate:
Do video models learn ``world models'' that discover laws of physics -- or,
alternatively, are they merely sophisticated pixel predictors that achieve
visual realism without understanding the physical principles of reality? We
address this question by developing Physics-IQ, a comprehensive benchmark
dataset that can only be solved by acquiring a deep understanding of various
physical principles, like fluid dynamics, optics, solid mechanics, magnetism
and thermodynamics. We find that across a range of current models (Sora,
Runway, Pika, Lumiere, Stable Video Diffusion, and VideoPoet), physical
understanding is severely limited, and unrelated to visual realism. At the same
time, some test cases can already be successfully solved. This indicates that
acquiring certain physical principles from observation alone may be possible,
but significant challenges remain. While we expect rapid advances ahead, our
work demonstrates that visual realism does not imply physical understanding.
Our project page is at https://physics-iq.github.io; code at
https://github.com/google-deepmind/physics-IQ-benchmark.
OmniThink: Expanding Knowledge Boundaries in Machine Writing through
  Thinking
Authors: name: Zekun Xi, name: Wenbiao Yin, name: Jizhan Fang, name: Jialong Wu, name: Runnan Fang, name: Ningyu Zhang, name: Jiang Yong, name: Pengjun Xie, name: Fei Huang, name: Huajun Chen
Published: 2025-01-16 18:58:06

Machine writing with large language models often relies on
retrieval-augmented generation. However, these approaches remain confined
within the boundaries of the model's predefined scope, limiting the generation
of content with rich information. Specifically, vanilla-retrieved information
tends to lack depth, utility, and suffers from redundancy, which negatively
impacts the quality of generated articles, leading to shallow, repetitive, and
unoriginal outputs. To address these issues, we propose OmniThink, a machine
writing framework that emulates the human-like process of iterative expansion
and reflection. The core idea behind OmniThink is to simulate the cognitive
behavior of learners as they progressively deepen their knowledge of the
topics. Experimental results demonstrate that OmniThink improves the knowledge
density of generated articles without compromising metrics such as coherence
and depth. Human evaluations and expert feedback further highlight the
potential of OmniThink to address real-world challenges in the generation of
long-form articles.
Exploring the Inquiry-Diagnosis Relationship with Advanced Patient
  Simulators
Authors: name: Zhaocheng Liu, name: Quan Tu, name: Wen Ye, name: Yu Xiao, name: Zhishou Zhang, name: Hengfu Cui, name: Yalun Zhu, name: Qiang Ju, name: Shizheng Li, name: Jian Xie
Published: 2025-01-16 11:41:14

Online medical consultation (OMC) restricts doctors to gathering patient
information solely through inquiries, making the already complex sequential
decision-making process of diagnosis even more challenging. Recently, the rapid
advancement of large language models has demonstrated a significant potential
to transform OMC. However, most studies have primarily focused on improving
diagnostic accuracy under conditions of relatively sufficient information,
while paying limited attention to the "inquiry" phase of the consultation
process. This lack of focus has left the relationship between "inquiry" and
"diagnosis" insufficiently explored. In this paper, we first extract real
patient interaction strategies from authentic doctor-patient conversations and
use these strategies to guide the training of a patient simulator that closely
mirrors real-world behavior. By inputting medical records into our patient
simulator to simulate patient responses, we conduct extensive experiments to
explore the relationship between "inquiry" and "diagnosis" in the consultation
process. Experimental results demonstrate that inquiry and diagnosis adhere to
the Liebig's law: poor inquiry quality limits the effectiveness of diagnosis,
regardless of diagnostic capability, and vice versa. Furthermore, the
experiments reveal significant differences in the inquiry performance of
various models. To investigate this phenomenon, we categorize the inquiry
process into four types: (1) chief complaint inquiry; (2) specification of
known symptoms; (3) inquiry about accompanying symptoms; and (4) gathering
family or medical history. We analyze the distribution of inquiries across the
four types for different models to explore the reasons behind their significant
performance differences. We plan to open-source the weights and related code of
our patient simulator at https://github.com/LIO-H-ZEN/PatientSimulator.
SynthLight: Portrait Relighting with Diffusion Model by Learning to
  Re-render Synthetic Faces
Authors: name: Sumit Chaturvedi, name: Mengwei Ren, name: Yannick Hold-Geoffroy, name: Jingyuan Liu, name: Julie Dorsey, name: Zhixin Shu
Published: 2025-01-16 18:59:48

We introduce SynthLight, a diffusion model for portrait relighting. Our
approach frames image relighting as a re-rendering problem, where pixels are
transformed in response to changes in environmental lighting conditions. Using
a physically-based rendering engine, we synthesize a dataset to simulate this
lighting-conditioned transformation with 3D head assets under varying lighting.
We propose two training and inference strategies to bridge the gap between the
synthetic and real image domains: (1) multi-task training that takes advantage
of real human portraits without lighting labels; (2) an inference time
diffusion sampling procedure based on classifier-free guidance that leverages
the input portrait to better preserve details. Our method generalizes to
diverse real photographs and produces realistic illumination effects, including
specular highlights and cast shadows, while preserving the subject's identity.
Our quantitative experiments on Light Stage data demonstrate results comparable
to state-of-the-art relighting methods. Our qualitative results on in-the-wild
images showcase rich and unprecedented illumination effects. Project Page:
https://vrroom.github.io/synthlight/
FAST: Efficient Action Tokenization for Vision-Language-Action Models
Authors: name: Karl Pertsch, name: Kyle Stachowicz, name: Brian Ichter, name: Danny Driess, name: Suraj Nair, name: Quan Vuong, name: Oier Mees, name: Chelsea Finn, name: Sergey Levine
Published: 2025-01-16 18:57:04

Autoregressive sequence models, such as Transformer-based vision-language
action (VLA) policies, can be tremendously effective for capturing complex and
generalizable robotic behaviors. However, such models require us to choose a
tokenization of our continuous action signals, which determines how the
discrete symbols predicted by the model map to continuous robot actions. We
find that current approaches for robot action tokenization, based on simple
per-dimension, per-timestep binning schemes, typically perform poorly when
learning dexterous skills from high-frequency robot data. To address this
challenge, we propose a new compression-based tokenization scheme for robot
actions, based on the discrete cosine transform. Our tokenization approach,
Frequency-space Action Sequence Tokenization (FAST), enables us to train
autoregressive VLAs for highly dexterous and high-frequency tasks where
standard discretization methods fail completely. Based on FAST, we release
FAST+, a universal robot action tokenizer, trained on 1M real robot action
trajectories. It can be used as a black-box tokenizer for a wide range of robot
action sequences, with diverse action spaces and control frequencies. Finally,
we show that, when combined with the pi0 VLA, our method can scale to training
on 10k hours of robot data and match the performance of diffusion VLAs, while
reducing training time by up to 5x.
AnyStory: Towards Unified Single and Multiple Subject Personalization in
  Text-to-Image Generation
Authors: name: Junjie He, name: Yuxiang Tuo, name: Binghui Chen, name: Chongyang Zhong, name: Yifeng Geng, name: Liefeng Bo
Published: 2025-01-16 12:28:39

Recently, large-scale generative models have demonstrated outstanding
text-to-image generation capabilities. However, generating high-fidelity
personalized images with specific subjects still presents challenges,
especially in cases involving multiple subjects. In this paper, we propose
AnyStory, a unified approach for personalized subject generation. AnyStory not
only achieves high-fidelity personalization for single subjects, but also for
multiple subjects, without sacrificing subject fidelity. Specifically, AnyStory
models the subject personalization problem in an "encode-then-route" manner. In
the encoding step, AnyStory utilizes a universal and powerful image encoder,
i.e., ReferenceNet, in conjunction with CLIP vision encoder to achieve
high-fidelity encoding of subject features. In the routing step, AnyStory
utilizes a decoupled instance-aware subject router to accurately perceive and
predict the potential location of the corresponding subject in the latent
space, and guide the injection of subject conditions. Detailed experimental
results demonstrate the excellent performance of our method in retaining
subject details, aligning text descriptions, and personalizing for multiple
subjects. The project page is at https://aigcdesigngroup.github.io/AnyStory/ .
CaPa: Carve-n-Paint Synthesis for Efficient 4K Textured Mesh Generation
Authors: name: Hwan Heo, name: Jangyeong Kim, name: Seongyeong Lee, name: Jeong A Wi, name: Junyoung Choi, name: Sangjun Ahn
Published: 2025-01-16 10:03:15

The synthesis of high-quality 3D assets from textual or visual inputs has
become a central objective in modern generative modeling. Despite the
proliferation of 3D generation algorithms, they frequently grapple with
challenges such as multi-view inconsistency, slow generation times, low
fidelity, and surface reconstruction problems. While some studies have
addressed some of these issues, a comprehensive solution remains elusive. In
this paper, we introduce CaPa, a carve-and-paint framework that
generates high-fidelity 3D assets efficiently. CaPa employs a two-stage
process, decoupling geometry generation from texture synthesis. Initially, a 3D
latent diffusion model generates geometry guided by multi-view inputs, ensuring
structural consistency across perspectives. Subsequently, leveraging a novel,
model-agnostic Spatially Decoupled Attention, the framework synthesizes
high-resolution textures (up to 4K) for a given geometry. Furthermore, we
propose a 3D-aware occlusion inpainting algorithm that fills untextured
regions, resulting in cohesive results across the entire model. This pipeline
generates high-quality 3D assets in less than 30 seconds, providing
ready-to-use outputs for commercial applications. Experimental results
demonstrate that CaPa excels in both texture fidelity and geometric stability,
establishing a new standard for practical, scalable 3D asset generation.
Learnings from Scaling Visual Tokenizers for Reconstruction and
  Generation
Authors: name: Philippe Hansen-Estruch, name: David Yan, name: Ching-Yao Chung, name: Orr Zohar, name: Jialiang Wang, name: Tingbo Hou, name: Tao Xu, name: Sriram Vishwanath, name: Peter Vajda, name: Xinlei Chen
Published: 2025-01-16 18:59:04

Visual tokenization via auto-encoding empowers state-of-the-art image and
video generative models by compressing pixels into a latent space. Although
scaling Transformer-based generators has been central to recent advances, the
tokenizer component itself is rarely scaled, leaving open questions about how
auto-encoder design choices influence both its objective of reconstruction and
downstream generative performance. Our work aims to conduct an exploration of
scaling in auto-encoders to fill in this blank. To facilitate this exploration,
we replace the typical convolutional backbone with an enhanced Vision
Transformer architecture for Tokenization (ViTok). We train ViTok on
large-scale image and video datasets far exceeding ImageNet-1K, removing data
constraints on tokenizer scaling. We first study how scaling the auto-encoder
bottleneck affects both reconstruction and generation -- and find that while it
is highly correlated with reconstruction, its relationship with generation is
more complex. We next explored the effect of separately scaling the
auto-encoders' encoder and decoder on reconstruction and generation
performance. Crucially, we find that scaling the encoder yields minimal gains
for either reconstruction or generation, while scaling the decoder boosts
reconstruction but the benefits for generation are mixed. Building on our
exploration, we design ViTok as a lightweight auto-encoder that achieves
competitive performance with state-of-the-art auto-encoders on ImageNet-1K and
COCO reconstruction tasks (256p and 512p) while outperforming existing
auto-encoders on 16-frame 128p video reconstruction for UCF-101, all with 2-5x
fewer FLOPs. When integrated with Diffusion Transformers, ViTok demonstrates
competitive performance on image generation for ImageNet-1K and sets new
state-of-the-art benchmarks for class-conditional video generation on UCF-101.
Inference-Time Scaling for Diffusion Models beyond Scaling Denoising
  Steps
Authors: name: Nanye Ma, name: Shangyuan Tong, name: Haolin Jia, name: Hexiang Hu, name: Yu-Chuan Su, name: Mingda Zhang, name: Xuan Yang, name: Yandong Li, name: Tommi Jaakkola, name: Xuhui Jia, name: Saining Xie
Published: 2025-01-16 18:30:37

Generative models have made significant impacts across various domains,
largely due to their ability to scale during training by increasing data,
computational resources, and model size, a phenomenon characterized by the
scaling laws. Recent research has begun to explore inference-time scaling
behavior in Large Language Models (LLMs), revealing how performance can further
improve with additional computation during inference. Unlike LLMs, diffusion
models inherently possess the flexibility to adjust inference-time computation
via the number of denoising steps, although the performance gains typically
flatten after a few dozen. In this work, we explore the inference-time scaling
behavior of diffusion models beyond increasing denoising steps and investigate
how the generation performance can further improve with increased computation.
Specifically, we consider a search problem aimed at identifying better noises
for the diffusion sampling process. We structure the design space along two
axes: the verifiers used to provide feedback, and the algorithms used to find
better noise candidates. Through extensive experiments on class-conditioned and
text-conditioned image generation benchmarks, our findings reveal that
increasing inference-time compute leads to substantial improvements in the
quality of samples generated by diffusion models, and with the complicated
nature of images, combinations of the components in the framework can be
specifically chosen to conform with different application scenario.
Towards Large Reasoning Models: A Survey of Reinforced Reasoning with
  Large Language Models
Authors: name: Fengli Xu, name: Qianyue Hao, name: Zefang Zong, name: Jingwei Wang, name: Yunke Zhang, name: Jingyi Wang, name: Xiaochong Lan, name: Jiahui Gong, name: Tianjian Ouyang, name: Fanjin Meng, name: Chenyang Shao, name: Yuwei Yan, name: Qinglong Yang, name: Yiwen Song, name: Sijian Ren, name: Xinyuan Hu, name: Yu Li, name: Jie Feng, name: Chen Gao, name: Yong Li
Published: 2025-01-16 17:37:58

Language has long been conceived as an essential tool for human reasoning.
The breakthrough of Large Language Models (LLMs) has sparked significant
research interest in leveraging these models to tackle complex reasoning tasks.
Researchers have moved beyond simple autoregressive token generation by
introducing the concept of "thought" -- a sequence of tokens representing
intermediate steps in the reasoning process. This innovative paradigm enables
LLMs' to mimic complex human reasoning processes, such as tree search and
reflective thinking. Recently, an emerging trend of learning to reason has
applied reinforcement learning (RL) to train LLMs to master reasoning
processes. This approach enables the automatic generation of high-quality
reasoning trajectories through trial-and-error search algorithms, significantly
expanding LLMs' reasoning capacity by providing substantially more training
data. Furthermore, recent studies demonstrate that encouraging LLMs to "think"
with more tokens during test-time inference can further significantly boost
reasoning accuracy. Therefore, the train-time and test-time scaling combined to
show a new research frontier -- a path toward Large Reasoning Model. The
introduction of OpenAI's o1 series marks a significant milestone in this
research direction. In this survey, we present a comprehensive review of recent
progress in LLM reasoning. We begin by introducing the foundational background
of LLMs and then explore the key technical components driving the development
of large reasoning models, with a focus on automated data construction,
learning-to-reason techniques, and test-time scaling. We also analyze popular
open-source projects at building large reasoning models, and conclude with open
challenges and future research directions.
RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation
Authors: name: Kaiqu Liang, name: Haimin Hu, name: Ryan Liu, name: Thomas L. Griffiths, name: Jaime Fernández Fisac
Published: 2025-01-15 06:33:15

Generative AI systems like foundation models (FMs) must align well with human
values to ensure their behavior is helpful and trustworthy. While Reinforcement
Learning from Human Feedback (RLHF) has shown promise for optimizing model
performance using human judgments, existing RLHF pipelines predominantly rely
on immediate feedback, which can fail to accurately reflect the downstream
impact of an interaction on users' utility. We demonstrate that feedback based
on evaluators' foresight estimates of downstream consequences systematically
induces Goodhart's Law dynamics, incentivizing misaligned behaviors like
sycophancy and deception and ultimately degrading user outcomes. To alleviate
this, we propose decoupling evaluation from prediction by refocusing RLHF on
hindsight feedback. Our theoretical analysis reveals that conditioning
evaluator feedback on downstream observations mitigates misalignment and
improves expected human utility, even when these observations are simulated by
the AI system itself. To leverage this insight in a practical alignment
algorithm, we introduce Reinforcement Learning from Hindsight Simulation
(RLHS), which first simulates plausible consequences and then elicits feedback
to assess what behaviors were genuinely beneficial in hindsight. We apply RLHS
to two widely-employed online and offline preference optimization methods --
Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO) --
and show empirically that misalignment is significantly reduced with both
methods. Through an online human user study, we show that RLHS consistently
outperforms RLHF in helping users achieve their goals and earns higher
satisfaction ratings, despite being trained solely with simulated hindsight
feedback. These results underscore the importance of focusing on long-term
consequences, even simulated ones, to mitigate misalignment in RLHF.