Orionnnn/hub-stats huggingface dataset:
Fast Encoder-Based 3D from Casual Videos via Point Track Processing
Authors: name: Yoni Kasten, name: Wuyue Lu, name: Haggai Maron
Published: 2024-04-10 15:37:00

This paper addresses the long-standing challenge of reconstructing 3D
structures from videos with dynamic content. Current approaches to this problem
were not designed to operate on casual videos recorded by standard cameras or
require a long optimization time.
  Aiming to significantly improve the efficiency of previous approaches, we
present TracksTo4D, a learning-based approach that enables inferring 3D
structure and camera positions from dynamic content originating from casual
videos using a single efficient feed-forward pass. To achieve this, we propose
operating directly over 2D point tracks as input and designing an architecture
tailored for processing 2D point tracks. Our proposed architecture is designed
with two key principles in mind: (1) it takes into account the inherent
symmetries present in the input point tracks data, and (2) it assumes that the
movement patterns can be effectively represented using a low-rank
approximation. TracksTo4D is trained in an unsupervised way on a dataset of
casual videos utilizing only the 2D point tracks extracted from the videos,
without any 3D supervision. Our experiments show that TracksTo4D can
reconstruct a temporal point cloud and camera positions of the underlying video
with accuracy comparable to state-of-the-art methods, while drastically
reducing runtime by up to 95\%. We further show that TracksTo4D generalizes
well to unseen videos of unseen semantic categories at inference time.
DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot
  Planning
Authors: name: Gaoyue Zhou, name: Hengkai Pan, name: Yann LeCun, name: Lerrel Pinto
Published: 2024-11-07 18:54:37

The ability to predict future outcomes given control actions is fundamental
for physical reasoning. However, such predictive models, often called world
models, have proven challenging to learn and are typically developed for
task-specific solutions with online policy learning. We argue that the true
potential of world models lies in their ability to reason and plan across
diverse problems using only passive data. Concretely, we require world models
to have the following three properties: 1) be trainable on offline,
pre-collected trajectories, 2) support test-time behavior optimization, and 3)
facilitate task-agnostic reasoning. To realize this, we present DINO World
Model (DINO-WM), a new method to model visual dynamics without reconstructing
the visual world. DINO-WM leverages spatial patch features pre-trained with
DINOv2, enabling it to learn from offline behavioral trajectories by predicting
future patch features. This design allows DINO-WM to achieve observational
goals through action sequence optimization, facilitating task-agnostic behavior
planning by treating desired goal patch features as prediction targets. We
evaluate DINO-WM across various domains, including maze navigation, tabletop
pushing, and particle manipulation. Our experiments demonstrate that DINO-WM
can generate zero-shot behavioral solutions at test time without relying on
expert demonstrations, reward modeling, or pre-learned inverse models. Notably,
DINO-WM exhibits strong generalization capabilities compared to prior
state-of-the-art work, adapting to diverse task families such as arbitrarily
configured mazes, push manipulation with varied object shapes, and
multi-particle scenarios.
Constitutional Classifiers: Defending against Universal Jailbreaks
  across Thousands of Hours of Red Teaming
Authors: name: Mrinank Sharma, name: Meg Tong, name: Jesse Mu, name: Jerry Wei, name: Jorrit Kruthoff, name: Scott Goodfriend, name: Euan Ong, name: Alwin Peng, name: Raj Agarwal, name: Cem Anil, name: Amanda Askell, name: Nathan Bailey, name: Joe Benton, name: Emma Bluemke, name: Samuel R. Bowman, name: Eric Christiansen, name: Hoagy Cunningham, name: Andy Dau, name: Anjali Gopal, name: Rob Gilson, name: Logan Graham, name: Logan Howard, name: Nimit Kalra, name: Taesung Lee, name: Kevin Lin, name: Peter Lofgren, name: Francesco Mosconi, name: Clare O'Hara, name: Catherine Olsson, name: Linda Petrini, name: Samir Rajani, name: Nikhil Saxena, name: Alex Silverstein, name: Tanya Singh, name: Theodore Sumers, name: Leonard Tang, name: Kevin K. Troy, name: Constantin Weisser, name: Ruiqi Zhong, name: Giulio Zhou, name: Jan Leike, name: Jared Kaplan, name: Ethan Perez
Published: 2025-01-31 01:09:32

Large language models (LLMs) are vulnerable to universal jailbreaks-prompting
strategies that systematically bypass model safeguards and enable users to
carry out harmful processes that require many model interactions, like
manufacturing illegal substances at scale. To defend against these attacks, we
introduce Constitutional Classifiers: safeguards trained on synthetic data,
generated by prompting LLMs with natural language rules (i.e., a constitution)
specifying permitted and restricted content. In over 3,000 estimated hours of
red teaming, no red teamer found a universal jailbreak that could extract
information from an early classifier-guarded LLM at a similar level of detail
to an unguarded model across most target queries. On automated evaluations,
enhanced classifiers demonstrated robust defense against held-out
domain-specific jailbreaks. These classifiers also maintain deployment
viability, with an absolute 0.38% increase in production-traffic refusals and a
23.7% inference overhead. Our work demonstrates that defending against
universal jailbreaks while maintaining practical deployment viability is
tractable.
Reward-Guided Speculative Decoding for Efficient LLM Reasoning
Authors: name: Baohao Liao, name: Yuhui Xu, name: Hanze Dong, name: Junnan Li, name: Christof Monz, name: Silvio Savarese, name: Doyen Sahoo, name: Caiming Xiong
Published: 2025-01-31 17:19:57

We introduce Reward-Guided Speculative Decoding (RSD), a novel framework
aimed at improving the efficiency of inference in large language models (LLMs).
RSD synergistically combines a lightweight draft model with a more powerful
target model, incorporating a controlled bias to prioritize high-reward
outputs, in contrast to existing speculative decoding methods that enforce
strict unbiasedness. RSD employs a process reward model to evaluate
intermediate decoding steps and dynamically decide whether to invoke the target
model, optimizing the trade-off between computational cost and output quality.
We theoretically demonstrate that a threshold-based mixture strategy achieves
an optimal balance between resource utilization and performance. Extensive
evaluations on challenging reasoning benchmarks, including Olympiad-level
tasks, show that RSD delivers significant efficiency gains against decoding
with the target model only (up to 4.4x fewer FLOPs), while achieving
significant better accuracy than parallel decoding method on average (up to
+3.5). These results highlight RSD as a robust and cost-effective approach for
deploying LLMs in resource-intensive scenarios.
s1: Simple test-time scaling
Authors: name: Niklas Muennighoff, name: Zitong Yang, name: Weijia Shi, name: Xiang Lisa Li, name: Li Fei-Fei, name: Hannaneh Hajishirzi, name: Luke Zettlemoyer, name: Percy Liang, name: Emmanuel Candès, name: Tatsunori Hashimoto
Published: 2025-01-31 18:48:08

Test-time scaling is a promising new approach to language modeling that uses
extra test-time compute to improve performance. Recently, OpenAI's o1 model
showed this capability but did not publicly share its methodology, leading to
many replication efforts. We seek the simplest approach to achieve test-time
scaling and strong reasoning performance. First, we curate a small dataset s1K
of 1,000 questions paired with reasoning traces relying on three criteria we
validate through ablations: difficulty, diversity, and quality. Second, we
develop budget forcing to control test-time compute by forcefully terminating
the model's thinking process or lengthening it by appending "Wait" multiple
times to the model's generation when it tries to end. This can lead the model
to double-check its answer, often fixing incorrect reasoning steps. After
supervised finetuning the Qwen2.5-32B-Instruct language model on s1K and
equipping it with budget forcing, our model s1 exceeds o1-preview on
competition math questions by up to 27% (MATH and AIME24). Further, scaling s1
with budget forcing allows extrapolating beyond its performance without
test-time intervention: from 50% to 57% on AIME24. Our model, data, and code
are open-source at https://github.com/simplescaling/s1.
Trading Inference-Time Compute for Adversarial Robustness
Authors: name: Wojciech Zaremba, name: Evgenia Nitishinskaya, name: Boaz Barak, name: Stephanie Lin, name: Sam Toyer, name: Yaodong Yu, name: Rachel Dias, name: Eric Wallace, name: Kai Xiao, name: Johannes Heidecke, name: Amelia Glaese
Published: 2025-01-31 01:20:44

We conduct experiments on the impact of increasing inference-time compute in
reasoning models (specifically OpenAI o1-preview and o1-mini) on their
robustness to adversarial attacks. We find that across a variety of attacks,
increased inference-time compute leads to improved robustness. In many cases
(with important exceptions), the fraction of model samples where the attack
succeeds tends to zero as the amount of test-time compute grows. We perform no
adversarial training for the tasks we study, and we increase inference-time
compute by simply allowing the models to spend more compute on reasoning,
independently of the form of attack. Our results suggest that inference-time
compute has the potential to improve adversarial robustness for Large Language
Models. We also explore new attacks directed at reasoning models, as well as
settings where inference-time compute does not improve reliability, and
speculate on the reasons for these as well as ways to address them.
SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute
  in Linear Diffusion Transformer
Authors: name: Enze Xie, name: Junsong Chen, name: Yuyang Zhao, name: Jincheng Yu, name: Ligeng Zhu, name: Yujun Lin, name: Zhekai Zhang, name: Muyang Li, name: Junyu Chen, name: Han Cai, name: Bingchen Liu, name: Daquan Zhou, name: Song Han
Published: 2025-01-30 15:31:48

This paper presents SANA-1.5, a linear Diffusion Transformer for efficient
scaling in text-to-image generation. Building upon SANA-1.0, we introduce three
key innovations: (1) Efficient Training Scaling: A depth-growth paradigm that
enables scaling from 1.6B to 4.8B parameters with significantly reduced
computational resources, combined with a memory-efficient 8-bit optimizer. (2)
Model Depth Pruning: A block importance analysis technique for efficient model
compression to arbitrary sizes with minimal quality loss. (3) Inference-time
Scaling: A repeated sampling strategy that trades computation for model
capacity, enabling smaller models to match larger model quality at inference
time. Through these strategies, SANA-1.5 achieves a text-image alignment score
of 0.72 on GenEval, which can be further improved to 0.80 through inference
scaling, establishing a new SoTA on GenEval benchmark. These innovations enable
efficient model scaling across different compute budgets while maintaining high
quality, making high-quality image generation more accessible.
CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web
  Navigation
Authors: name: Faria Huq, name: Zora Zhiruo Wang, name: Frank F. Xu, name: Tianyue Ou, name: Shuyan Zhou, name: Jeffrey P. Bigham, name: Graham Neubig
Published: 2025-01-28 00:56:53

While much work on web agents emphasizes the promise of autonomously
performing tasks on behalf of users, in reality, agents often fall short on
complex tasks in real-world contexts and modeling user preference. This
presents an opportunity for humans to collaborate with the agent and leverage
the agent's capabilities effectively. We propose CowPilot, a framework
supporting autonomous as well as human-agent collaborative web navigation, and
evaluation across task success and task efficiency. CowPilot reduces the number
of steps humans need to perform by allowing agents to propose next steps, while
users are able to pause, reject, or take alternative actions. During execution,
users can interleave their actions with the agent by overriding suggestions or
resuming agent control when needed. We conducted case studies on five common
websites and found that the human-agent collaborative mode achieves the highest
success rate of 95% while requiring humans to perform only 15.2% of the total
steps. Even with human interventions during task execution, the agent
successfully drives up to half of task success on its own. CowPilot can serve
as a useful tool for data collection and agent evaluation across websites,
which we believe will enable research in how users and agents can work
together. Video demonstrations are available at
https://oaishi.github.io/cowpilot.html
Streaming DiLoCo with overlapping communication: Towards a Distributed
  Free Lunch
Authors: name: Arthur Douillard, name: Yanislav Donchev, name: Keith Rush, name: Satyen Kale, name: Zachary Charles, name: Zachary Garrett, name: Gabriel Teston, name: Dave Lacey, name: Ross McIlroy, name: Jiajun Shen, name: Alexandre Ramé, name: Arthur Szlam, name: Marc'Aurelio Ranzato, name: Paul Barham
Published: 2025-01-30 17:23:50

Training of large language models (LLMs) is typically distributed across a
large number of accelerators to reduce training time. Since internal states and
parameter gradients need to be exchanged at each and every single gradient
step, all devices need to be co-located using low-latency high-bandwidth
communication links to support the required high volume of exchanged bits.
Recently, distributed algorithms like DiLoCo have relaxed such co-location
constraint: accelerators can be grouped into ``workers'', where
synchronizations between workers only occur infrequently. This in turn means
that workers can afford being connected by lower bandwidth communication links
without affecting learning quality. However, in these methods, communication
across workers still requires the same peak bandwidth as before, as the
synchronizations require all parameters to be exchanged across all workers. In
this paper, we improve DiLoCo in three ways. First, we synchronize only subsets
of parameters in sequence, rather than all at once, which greatly reduces peak
bandwidth. Second, we allow workers to continue training while synchronizing,
which decreases wall clock time. Third, we quantize the data exchanged by
workers, which further reduces bandwidth across workers. By properly combining
these modifications, we show experimentally that we can distribute training of
billion-scale parameters and reach similar quality as before, but reducing
required bandwidth by two orders of magnitude.
MedXpertQA: Benchmarking Expert-Level Medical Reasoning and
  Understanding
Authors: name: Yuxin Zuo, name: Shang Qu, name: Yifei Li, name: Zhangren Chen, name: Xuekai Zhu, name: Ermo Hua, name: Kaiyan Zhang, name: Ning Ding, name: Bowen Zhou
Published: 2025-01-30 14:07:56

We introduce MedXpertQA, a highly challenging and comprehensive benchmark to
evaluate expert-level medical knowledge and advanced reasoning. MedXpertQA
includes 4,460 questions spanning 17 specialties and 11 body systems. It
includes two subsets, Text for text evaluation and MM for multimodal
evaluation. Notably, MM introduces expert-level exam questions with diverse
images and rich clinical information, including patient records and examination
results, setting it apart from traditional medical multimodal benchmarks with
simple QA pairs generated from image captions. MedXpertQA applies rigorous
filtering and augmentation to address the insufficient difficulty of existing
benchmarks like MedQA, and incorporates specialty board questions to improve
clinical relevance and comprehensiveness. We perform data synthesis to mitigate
data leakage risk and conduct multiple rounds of expert reviews to ensure
accuracy and reliability. We evaluate 16 leading models on MedXpertQA.
Moreover, medicine is deeply connected to real-world decision-making, providing
a rich and representative setting for assessing reasoning abilities beyond
mathematics and code. To this end, we develop a reasoning-oriented subset to
facilitate the assessment of o1-like models.
WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in
  Post-Training
Authors: name: Benjamin Feuer, name: Chinmay Hegde
Published: 2025-01-30 17:21:44

Language model (LLM) post-training, from DPO to distillation, can refine
behaviors and unlock new skills, but the open science supporting these
post-training techniques is still in its infancy. One limiting factor has been
the difficulty of conducting large-scale comparative analyses of synthetic data
generating models and LLM judges. To close this gap, we introduce WILDCHAT-50M,
the largest public chat dataset to date. We extend the existing WildChat
dataset to include responses not only from GPT, but from over 50 different
open-weight models, ranging in size from 0.5B to 104B parameters. We conduct an
extensive comparative analysis and demonstrate the potential of this dataset by
creating RE-WILD, our own public SFT mix, which outperforms the recent Tulu-3
SFT mixture from Allen AI with only 40% as many samples. Our dataset, samples
and code are available at https://github.com/penfever/wildchat-50m.
o3-mini vs DeepSeek-R1: Which One is Safer?
Authors: name: Aitor Arrieta, name: Miriam Ugarte, name: Pablo Valle, name: José Antonio Parejo, name: Sergio Segura
Published: 2025-01-30 15:45:56

The irruption of DeepSeek-R1 constitutes a turning point for the AI industry
in general and the LLMs in particular. Its capabilities have demonstrated
outstanding performance in several tasks, including creative thinking, code
generation, maths and automated program repair, at apparently lower execution
cost. However, LLMs must adhere to an important qualitative property, i.e.,
their alignment with safety and human values. A clear competitor of DeepSeek-R1
is its American counterpart, OpenAI's o3-mini model, which is expected to set
high standards in terms of performance, safety and cost. In this paper we
conduct a systematic assessment of the safety level of both, DeepSeek-R1 (70b
version) and OpenAI's o3-mini (beta version). To this end, we make use of our
recently released automated safety testing tool, named ASTRAL. By leveraging
this tool, we automatically and systematically generate and execute a total of
1260 unsafe test inputs on both models. After conducting a semi-automated
assessment of the outcomes provided by both LLMs, the results indicate that
DeepSeek-R1 is highly unsafe as compared to OpenAI's o3-mini. Based on our
evaluation, DeepSeek-R1 answered unsafely to 11.98% of the executed prompts
whereas o3-mini only to 1.19%.
Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
Authors: name: Yue Wang, name: Qiuzhi Liu, name: Jiahao Xu, name: Tian Liang, name: Xingyu Chen, name: Zhiwei He, name: Linfeng Song, name: Dian Yu, name: Juntao Li, name: Zhuosheng Zhang, name: Rui Wang, name: Zhaopeng Tu, name: Haitao Mi, name: Dong Yu
Published: 2025-01-30 18:58:18

Large language models (LLMs) such as OpenAI's o1 have demonstrated remarkable
abilities in complex reasoning tasks by scaling test-time compute and
exhibiting human-like deep thinking. However, we identify a phenomenon we term
underthinking, where o1-like LLMs frequently switch between different reasoning
thoughts without sufficiently exploring promising paths to reach a correct
solution. This behavior leads to inadequate depth of reasoning and decreased
performance, particularly on challenging mathematical problems. To
systematically analyze this issue, we conduct experiments on three challenging
test sets and two representative open-source o1-like models, revealing that
frequent thought switching correlates with incorrect responses. We introduce a
novel metric to quantify underthinking by measuring token efficiency in
incorrect answers. To address underthinking, we propose a decoding strategy
with thought switching penalty TIP that discourages premature transitions
between thoughts, encouraging deeper exploration of each reasoning path.
Experimental results demonstrate that our approach improves accuracy across
challenging datasets without requiring model fine-tuning. Our findings
contribute to understanding reasoning inefficiencies in o1-like LLMs and offer
a practical solution to enhance their problem-solving capabilities.
Large Language Models Think Too Fast To Explore Effectively
Authors: name: Lan Pan, name: Hanbo Xie, name: Robert C. Wilson
Published: 2025-01-29 21:51:17

Large Language Models have emerged many intellectual capacities. While
numerous benchmarks assess their intelligence, limited attention has been given
to their ability to explore, an essential capacity for discovering new
information and adapting to novel environments in both natural and artificial
systems. The extent to which LLMs can effectively explore, particularly in
open-ended tasks, remains unclear. This study investigates whether LLMs can
surpass humans in exploration during an open-ended task, using Little Alchemy 2
as a paradigm, where agents combine elements to discover new ones. Results show
most LLMs underperform compared to humans, except for the o1 model, with those
traditional LLMs relying primarily on uncertainty driven strategies, unlike
humans who balance uncertainty and empowerment. Representational analysis of
the models with Sparse Autoencoders revealed that uncertainty and choices are
represented at earlier transformer blocks, while empowerment values are
processed later, causing LLMs to think too fast and make premature decisions,
hindering effective exploration. These findings shed light on the limitations
of LLM exploration and suggest directions for improving their adaptability.
PhysBench: Benchmarking and Enhancing Vision-Language Models for
  Physical World Understanding
Authors: name: Wei Chow, name: Jiageng Mao, name: Boyi Li, name: Daniel Seita, name: Vitor Guizilini, name: Yue Wang
Published: 2025-01-27 18:59:58

Understanding the physical world is a fundamental challenge in embodied AI,
critical for enabling agents to perform complex tasks and operate safely in
real-world environments. While Vision-Language Models (VLMs) have shown great
promise in reasoning and task planning for embodied agents, their ability to
comprehend physical phenomena remains extremely limited. To close this gap, we
introduce PhysBench, a comprehensive benchmark designed to evaluate VLMs'
physical world understanding capability across a diverse set of tasks.
PhysBench contains 10,002 entries of interleaved video-image-text data,
categorized into four major domains: physical object properties, physical
object relationships, physical scene understanding, and physics-based dynamics,
further divided into 19 subclasses and 8 distinct capability dimensions. Our
extensive experiments, conducted on 75 representative VLMs, reveal that while
these models excel in common-sense reasoning, they struggle with understanding
the physical world -- likely due to the absence of physical knowledge in their
training data and the lack of embedded physical priors. To tackle the
shortfall, we introduce PhysAgent, a novel framework that combines the
generalization strengths of VLMs with the specialized expertise of vision
models, significantly enhancing VLMs' physical understanding across a variety
of tasks, including an 18.4\% improvement on GPT-4o. Furthermore, our results
demonstrate that enhancing VLMs' physical world understanding capabilities can
help embodied agents such as MOKA. We believe that PhysBench and PhysAgent
offer valuable insights and contribute to bridging the gap between VLMs and
physical world understanding.