Orionnnn/hub-stats huggingface dataset:
Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modeling
Authors: name: Mingtong Zhang, name: Kaifeng Zhang, name: Yunzhu Li
Published: 2024-10-24 17:02:52

Videos of robots interacting with objects encode rich information about the
objects' dynamics. However, existing video prediction approaches typically do
not explicitly account for the 3D information from videos, such as robot
actions and objects' 3D states, limiting their use in real-world robotic
applications. In this work, we introduce a framework to learn object dynamics
directly from multi-view RGB videos by explicitly considering the robot's
action trajectories and their effects on scene dynamics. We utilize the 3D
Gaussian representation of 3D Gaussian Splatting (3DGS) to train a
particle-based dynamics model using Graph Neural Networks. This model operates
on sparse control particles downsampled from the densely tracked 3D Gaussian
reconstructions. By learning the neural dynamics model on offline robot
interaction data, our method can predict object motions under varying initial
configurations and unseen robot actions. The 3D transformations of Gaussians
can be interpolated from the motions of control particles, enabling the
rendering of predicted future object states and achieving action-conditioned
video prediction. The dynamics model can also be applied to model-based
planning frameworks for object manipulation tasks. We conduct experiments on
various kinds of deformable materials, including ropes, clothes, and stuffed
animals, demonstrating our framework's ability to model complex shapes and
dynamics. Our project page is available at https://gs-dynamics.github.io.
Reflection-Bench: probing AI intelligence with reflection
Authors: name: Lingyu Li, name: Yixu Wang, name: Haiquan Zhao, name: Shuqi Kong, name: Yan Teng, name: Chunbo Li, name: Yingchun Wang
Published: 2024-10-21 17:59:50

The ability to adapt beliefs or behaviors in response to unexpected outcomes,
reflection, is fundamental to intelligent systems' interaction with the world.
From a cognitive science perspective, this serves as a core principle of
intelligence applicable to both human and AI systems. To address the debate on
the intelligence of large language models (LLMs), we propose Reflection-Bench,
a comprehensive benchmark comprising 7 tasks spanning core cognitive functions
crucial for reflection, including perception, memory, belief updating,
decision-making, prediction, counterfactual thinking, and meta-reflection. We
evaluate the performances of 13 prominent LLMs such as OpenAI o1, GPT-4, Claude
3.5 Sonnet, etc. The results indicate that current LLMs still lack satisfactory
reflection ability. We discuss the underlying causes of these results and
suggest potential avenues for future research. In conclusion, Reflection-Bench
offers both evaluation tools and inspiration for developing AI capable of
reliably interacting with the environment. Our data and code are available at
https://github.com/YabYum/ReflectionBench.
Continuous Speech Synthesis using per-token Latent Diffusion
Authors: name: Arnon Turetzky, name: Nimrod Shabtay, name: Slava Shechtman, name: Hagai Aronowitz, name: David Haws, name: Ron Hoory, name: Avihu Dekel
Published: 2024-10-21 14:23:46

The success of autoregressive transformer models with discrete tokens has
inspired quantization-based approaches for continuous modalities, though these
often limit reconstruction quality. We therefore introduce SALAD, a per-token
latent diffusion model for zero-shot text-to-speech, that operates on
continuous representations. SALAD builds upon the recently proposed expressive
diffusion head for image generation, and extends it to generate variable-length
outputs. Our approach utilizes semantic tokens for providing contextual
information and determining the stopping condition. We suggest three continuous
variants for our method, extending popular discrete speech synthesis
techniques. Additionally, we implement discrete baselines for each variant and
conduct a comparative analysis of discrete versus continuous speech modeling
techniques. Our results demonstrate that both continuous and discrete
approaches are highly competent, and that SALAD achieves a superior
intelligibility score while obtaining speech quality and speaker similarity on
par with the ground-truth audio.
Counting Ability of Large Language Models and Impact of Tokenization
Authors: name: Xiang Zhang, name: Juntai Cao, name: Chenyu You
Published: 2024-10-25 17:56:24

Transformers, the backbone of modern large language models (LLMs), face
inherent architectural limitations that impede their reasoning capabilities.
Unlike recurrent networks, Transformers lack recurrent connections, confining
them to constant-depth computation. This restriction places them in the
complexity class TC^0, making them theoretically incapable of solving tasks
that demand increasingly deep reasoning as input length grows. Counting, a
fundamental component of many reasoning tasks, also requires reasoning depth to
grow linearly to be performed inductively. While previous studies have
established the upper limits of counting ability in Transformer-based expert
models (i.e., models specifically trained for counting tasks), these findings
do not directly extend to general-purpose LLMs due to differences in reasoning
mechanisms. Recent work has highlighted how Chain of Thought (CoT) reasoning
can help alleviate some of the architectural limitations of Transformers in
counting tasks. However, little attention has been paid to the role of
tokenization in these models. Unlike expert models that often use
character-level tokenization, LLMs typically rely on byte-level (BPE)
tokenizers, which fundamentally alters the way reasoning is processed. Our work
investigates the impact of tokenization on the counting abilities of LLMs,
uncovering substantial performance variations based on input tokenization
differences. We provide both theoretical and experimental analyses, offering
insights into how tokenization choices can undermine models' theoretical
computability, thereby inspiring the design of new tokenization methods to
enhance reasoning in LLMs.
ROCKET-1: Master Open-World Interaction with Visual-Temporal Context
  Prompting
Authors: name: Shaofei Cai, name: Zihao Wang, name: Kewei Lian, name: Zhancun Mu, name: Xiaojian Ma, name: Anji Liu, name: Yitao Liang
Published: 2024-10-23 13:26:59

Vision-language models (VLMs) have excelled in multimodal tasks, but adapting
them to embodied decision-making in open-world environments presents
challenges. A key issue is the difficulty in smoothly connecting individual
entities in low-level observations with abstract concepts required for
planning. A common approach to address this problem is through the use of
hierarchical agents, where VLMs serve as high-level reasoners that break down
tasks into executable sub-tasks, typically specified using language and
imagined observations. However, language often fails to effectively convey
spatial information, while generating future images with sufficient accuracy
remains challenging. To address these limitations, we propose visual-temporal
context prompting, a novel communication protocol between VLMs and policy
models. This protocol leverages object segmentation from both past and present
observations to guide policy-environment interactions. Using this approach, we
train ROCKET-1, a low-level policy that predicts actions based on concatenated
visual observations and segmentation masks, with real-time object tracking
provided by SAM-2. Our method unlocks the full potential of VLMs
visual-language reasoning abilities, enabling them to solve complex creative
tasks, especially those heavily reliant on spatial understanding. Experiments
in Minecraft demonstrate that our approach allows agents to accomplish
previously unattainable tasks, highlighting the effectiveness of
visual-temporal context prompting in embodied decision-making. Codes and demos
will be available on the project page: https://craftjarvis.github.io/ROCKET-1.
Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite
  Learning
Authors: name: Yujian Liu, name: Shiyu Chang, name: Tommi Jaakkola, name: Yang Zhang
Published: 2024-10-25 03:48:51

Recent studies have identified one aggravating factor of LLM hallucinations
as the knowledge inconsistency between pre-training and fine-tuning, where
unfamiliar fine-tuning data mislead the LLM to fabricate plausible but wrong
outputs. In this paper, we propose a novel fine-tuning strategy called
Prereq-Tune to address this knowledge inconsistency and reduce hallucinations.
Fundamentally, Prereq-Tune disentangles the learning of skills and knowledge,
so the model learns only the task skills without being impacted by the
knowledge inconsistency. To achieve this, Prereq-Tune introduces an additional
prerequisite learning stage to learn the necessary knowledge for SFT, allowing
subsequent SFT to focus only on task skills. Prereq-Tune can also be combined
with fictitious synthetic data to enhance the grounding of LLM outputs to their
internal knowledge. Experiments show that Prereq-Tune outperforms existing
baselines in improving LLM's factuality across short QA and long-form
generation tasks. It also opens new possibilities for knowledge-controlled
generation in LLMs. Our code is available at
https://github.com/UCSB-NLP-Chang/Prereq_tune.git.
Hybrid Preferences: Learning to Route Instances for Human vs. AI
  Feedback
Authors: name: Lester James V. Miranda, name: Yizhong Wang, name: Yanai Elazar, name: Sachin Kumar, name: Valentina Pyatkin, name: Faeze Brahman, name: Noah A. Smith, name: Hannaneh Hajishirzi, name: Pradeep Dasigi
Published: 2024-10-24 20:04:15

Learning from human feedback has enabled the alignment of language models
(LMs) with human preferences. However, directly collecting human preferences
can be expensive, time-consuming, and can have high variance. An appealing
alternative is to distill preferences from LMs as a source of synthetic
annotations as they are more consistent, cheaper, and scale better than human
annotation; however, they are also prone to biases and errors. In this work, we
introduce a routing framework that combines inputs from humans and LMs to
achieve better annotation quality, while reducing the total cost of human
annotation. The crux of our approach is to identify preference instances that
will benefit from human annotations. We formulate this as an optimization
problem: given a preference dataset and an evaluation metric, we train a
performance prediction model to predict a reward model's performance on an
arbitrary combination of human and LM annotations and employ a routing strategy
that selects a combination that maximizes predicted performance. We train the
performance prediction model on MultiPref, a new preference dataset with 10K
instances paired with human and LM labels. We show that the selected hybrid
mixture of LM and direct human preferences using our routing framework achieves
better reward model performance compared to using either one exclusively. We
simulate selective human preference collection on three other datasets and show
that our method generalizes well to all three. We analyze features from the
routing model to identify characteristics of instances that can benefit from
human feedback, e.g., prompts with a moderate safety concern or moderate intent
complexity. We release the dataset, annotation platform, and source code used
in this study to foster more efficient and accurate preference collection in
the future.
FasterCache: Training-Free Video Diffusion Model Acceleration with High
  Quality
Authors: name: Zhengyao Lv, name: Chenyang Si, name: Junhao Song, name: Zhenyu Yang, name: Yu Qiao, name: Ziwei Liu, name: Kwan-Yee K. Wong
Published: 2024-10-25 07:24:38

In this paper, we present \textit{FasterCache}, a novel
training-free strategy designed to accelerate the inference of video diffusion
models with high-quality generation. By analyzing existing cache-based methods,
we observe that directly reusing adjacent-step features degrades video
quality due to the loss of subtle variations. We further perform a pioneering
investigation of the acceleration potential of classifier-free guidance (CFG)
and reveal significant redundancy between conditional and unconditional
features within the same timestep. Capitalizing on these observations, we
introduce FasterCache to substantially accelerate diffusion-based video
generation. Our key contributions include a dynamic feature reuse strategy that
preserves both feature distinction and temporal continuity, and CFG-Cache which
optimizes the reuse of conditional and unconditional outputs to further enhance
inference speed without compromising video quality. We empirically evaluate
FasterCache on recent video diffusion models. Experimental results show that
FasterCache can significantly accelerate video generation (\eg 1.67times
speedup on Vchitect-2.0) while keeping video quality comparable to the
baseline, and consistently outperform existing methods in both inference speed
and video quality.
Leveraging Skills from Unlabeled Prior Data for Efficient Online
  Exploration
Authors: name: Max Wilcoxson, name: Qiyang Li, name: Kevin Frans, name: Sergey Levine
Published: 2024-10-23 17:58:45

Unsupervised pretraining has been transformative in many supervised domains.
However, applying such ideas to reinforcement learning (RL) presents a unique
challenge in that fine-tuning does not involve mimicking task-specific data,
but rather exploring and locating the solution through iterative
self-improvement. In this work, we study how unlabeled prior trajectory data
can be leveraged to learn efficient exploration strategies. While prior data
can be used to pretrain a set of low-level skills, or as additional off-policy
data for online RL, it has been unclear how to combine these ideas effectively
for online exploration. Our method SUPE (Skills from Unlabeled Prior data for
Exploration) demonstrates that a careful combination of these ideas compounds
their benefits. Our method first extracts low-level skills using a variational
autoencoder (VAE), and then pseudo-relabels unlabeled trajectories using an
optimistic reward model, transforming prior data into high-level, task-relevant
examples. Finally, SUPE uses these transformed examples as additional
off-policy data for online RL to learn a high-level policy that composes
pretrained low-level skills to explore efficiently. We empirically show that
SUPE reliably outperforms prior strategies, successfully solving a suite of
long-horizon, sparse-reward tasks. Code: https://github.com/rail-berkeley/supe.
Infinity-MM: Scaling Multimodal Performance with Large-Scale and
  High-Quality Instruction Data
Authors: name: Shuhao Gu, name: Jialing Zhang, name: Siyuan Zhou, name: Kevin Yu, name: Zhaohu Xing, name: Liangdong Wang, name: Zhou Cao, name: Jintao Jia, name: Zhuoyi Zhang, name: Yixuan Wang, name: Zhenchong Hu, name: Bo-Wen Zhang, name: Jijie Li, name: Dong Liang, name: Yingli Zhao, name: Yulong Ao, name: Yaoqi Liu, name: Fangxiang Feng, name: Guang Liu
Published: 2024-10-24 09:03:48

Vision-Language Models (VLMs) have recently made significant progress, but
the limited scale and quality of open-source instruction data hinder their
performance compared to closed-source models. In this work, we address this
limitation by introducing Infinity-MM, a large-scale multimodal instruction
dataset with 40 million samples, enhanced through rigorous quality filtering
and deduplication. We also propose a synthetic instruction generation method
based on open-source VLMs, using detailed image annotations and diverse
question generation. Using this data, we trained a 2-billion-parameter VLM,
Aquila-VL-2B, achieving state-of-the-art (SOTA) performance for models of
similar scale. This demonstrates that expanding instruction data and generating
synthetic data can significantly improve the performance of open-source models.
MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark
Authors: name: S Sakshi, name: Utkarsh Tyagi, name: Sonal Kumar, name: Ashish Seth, name: Ramaneswaran Selvakumar, name: Oriol Nieto, name: Ramani Duraiswami, name: Sreyan Ghosh, name: Dinesh Manocha
Published: 2024-10-24 21:20:10

The ability to comprehend audio--which includes speech, non-speech sounds,
and music--is crucial for AI agents to interact effectively with the world. We
present MMAU, a novel benchmark designed to evaluate multimodal audio
understanding models on tasks requiring expert-level knowledge and complex
reasoning. MMAU comprises 10k carefully curated audio clips paired with
human-annotated natural language questions and answers spanning speech,
environmental sounds, and music. It includes information extraction and
reasoning questions, requiring models to demonstrate 27 distinct skills across
unique and challenging tasks. Unlike existing benchmarks, MMAU emphasizes
advanced perception and reasoning with domain-specific knowledge, challenging
models to tackle tasks akin to those faced by experts. We assess 18 open-source
and proprietary (Large) Audio-Language Models, demonstrating the significant
challenges posed by MMAU. Notably, even the most advanced Gemini Pro v1.5
achieves only 52.97% accuracy, and the state-of-the-art open-source Qwen2-Audio
achieves only 52.50%, highlighting considerable room for improvement. We
believe MMAU will drive the audio and multimodal research community to develop
more advanced audio understanding models capable of solving complex audio
tasks.
Teach Multimodal LLMs to Comprehend Electrocardiographic Images
Authors: name: Ruoqi Liu, name: Yuelin Bai, name: Xiang Yue, name: Ping Zhang
Published: 2024-10-21 20:26:41

The electrocardiogram (ECG) is an essential non-invasive diagnostic tool for
assessing cardiac conditions. Existing automatic interpretation methods suffer
from limited generalizability, focusing on a narrow range of cardiac
conditions, and typically depend on raw physiological signals, which may not be
readily available in resource-limited settings where only printed or digital
ECG images are accessible. Recent advancements in multimodal large language
models (MLLMs) present promising opportunities for addressing these challenges.
However, the application of MLLMs to ECG image interpretation remains
challenging due to the lack of instruction tuning datasets and well-established
ECG image benchmarks for quantitative evaluation. To address these challenges,
we introduce ECGInstruct, a comprehensive ECG image instruction tuning dataset
of over one million samples, covering a wide range of ECG-related tasks from
diverse data sources. Using ECGInstruct, we develop PULSE, an MLLM tailored for
ECG image comprehension. In addition, we curate ECGBench, a new evaluation
benchmark covering four key ECG image interpretation tasks across nine
different datasets. Our experiments show that PULSE sets a new
state-of-the-art, outperforming general MLLMs with an average accuracy
improvement of 15% to 30%. This work highlights the potential of PULSE to
enhance ECG interpretation in clinical practice.
Pantograph: A Machine-to-Machine Interaction Interface for Advanced
  Theorem Proving, High Level Reasoning, and Data Extraction in Lean 4
Authors: name: Leni Aniva, name: Chuyue Sun, name: Brando Miranda, name: Clark Barrett, name: Sanmi Koyejo
Published: 2024-10-21 18:49:28

Machine-assisted theorem proving refers to the process of conducting
structured reasoning to automatically generate proofs for mathematical
theorems. Recently, there has been a surge of interest in using machine
learning models in conjunction with proof assistants to perform this task. In
this paper, we introduce Pantograph, a tool that provides a versatile interface
to the Lean 4 proof assistant and enables efficient proof search via powerful
search algorithms such as Monte Carlo Tree Search. In addition, Pantograph
enables high-level reasoning by enabling a more robust handling of Lean 4's
inference steps. We provide an overview of Pantograph's architecture and
features. We also report on an illustrative use case: using machine learning
models and proof sketches to prove Lean 4 theorems. Pantograph's innovative
features pave the way for more advanced machine learning models to perform
complex proof searches and high-level reasoning, equipping future researchers
to design more versatile and powerful theorem provers.
ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment
Authors: name: Elyas Obbad, name: Iddah Mlauzi, name: Brando Miranda, name: Rylan Schaeffer, name: Kamal Obbad, name: Suhana Bedi, name: Sanmi Koyejo
Published: 2024-10-23 18:01:06

Data selection is crucial for optimizing language model (LM) performance on
specific tasks, yet most existing methods fail to effectively consider the
target task distribution.
  Current approaches either ignore task-specific requirements entirely or rely
on approximations that fail to capture the nuanced patterns needed for tasks
like Autoformalization or code generation.
  Methods that do consider the target distribution often rely on simplistic,
sometimes noisy, representations, like hashed n-gram features, which can lead
to collisions and introduce noise.
  We introduce ZIP-FIT, a data selection framework that uses gzip compression
to directly measure alignment between potential training data and the target
task distribution.
  In extensive evaluations on Autoformalization and Python code generation,
ZIP-FIT significantly outperforms leading baselines like DSIR and D4.
  Models trained on ZIP-FIT-selected data achieve their lowest cross-entropy
loss up to 85.1\% faster than baselines, demonstrating that better task
alignment leads to more efficient learning.
  In addition, ZIP-FIT performs selection up to 65.8\% faster than DSIR and two
orders of magnitude faster than D4.
  Notably, ZIP-FIT shows that smaller, well-aligned datasets often outperform
larger but less targeted ones, demonstrating that a small amount of higher
quality data is superior to a large amount of lower quality data.
  Our results imply that task-aware data selection is crucial for efficient
domain adaptation, and that compression offers a principled way to measure task
alignment.
  By showing that targeted data selection can dramatically improve
task-specific performance, our work provides new insights into the relationship
between data quality, task alignment, and model learning efficiency.
The Nature of Mathematical Modeling and Probabilistic Optimization
  Engineering in Generative AI
Authors: name: Fulu Li
Published: 2024-10-24 05:29:20

In this paper, we give an in-depth analysis on the mathematical problem
formulations and the probabilistic optimization explorations for some of the
key components in Transformer model [33] in the field of generative AI. We
explore and discuss some potential further enhancement for current state of the
art methods for some key underlying technologies of generative AI models from
algorithmic and probabilistic optimization perspective. In particular, we
present an optimal solution for sub-word encoding (SWE) based on similar
initial settings as that of byte-pair encoding (BPE) algorithm in [9] with
similar objectives as that of WordPiece approach in [28, 31] to maximize the
likelihood of the training data. We also present cross entropy optimization
method to optimize hyperparameters for word2vec model [17]. In addition, we
propose a factored combination of rotary positional encoding (RoPE) [32] and
attention with linear biases (ALiBi) [23] with a harmonic series. We also
present a probabilistic FlashAttention [6, 7] (PrFlashAttention) method with a
probability distribution over block distances in the matrix to decide which
block is likely to participate in a given round of attention computation while
maintaining the lower triangle shape of the tensor for autoregressive language
models by re-shaping the tensors. Finally, we present staircase adaptive
quantization (SAQ) of key-value (KV) cache for multi-query attention (MQA)
based on the framework presented in [16] to have gradual quantization
degradation while achieving reasonable model quality and cost savings.