Orionnnn/hub-stats huggingface dataset:
B-STaR: Monitoring and Balancing Exploration and Exploitation in
  Self-Taught Reasoners
Authors: name: Weihao Zeng, name: Yuzhen Huang, name: Lulu Zhao, name: Yijun Wang, name: Zifei Shan, name: Junxian He
Published: 2024-12-23 03:58:34

In the absence of extensive human-annotated data for complex reasoning tasks,
self-improvement -- where models are trained on their own outputs -- has
emerged as a primary method for enhancing performance. However, the critical
factors underlying the mechanism of these iterative self-improving methods
remain poorly understood, such as under what conditions self-improvement is
effective, and what are the bottlenecks in the current iterations. In this
work, we identify and propose methods to monitor two pivotal factors in this
iterative process: (1) the model's ability to generate sufficiently diverse
responses (exploration); and (2) the effectiveness of external rewards in
distinguishing high-quality candidates from lower-quality ones (exploitation).
Using mathematical reasoning as a case study, we begin with a quantitative
analysis to track the dynamics of exploration and exploitation, discovering
that a model's exploratory capabilities rapidly deteriorate over iterations,
and the effectiveness of exploiting external rewards diminishes as well.
Motivated by these findings, we introduce B-STaR, a Self-Taught Reasoning
framework that autonomously adjusts configurations across iterations to Balance
exploration and exploitation, thereby optimizing the self-improving
effectiveness based on the current policy model and available rewards. Our
experiments on mathematical reasoning, coding, and commonsense reasoning
demonstrate that B-STaR not only enhances the model's exploratory capabilities
throughout training but also achieves a more effective balance between
exploration and exploitation, leading to superior performance.
RobustFT: Robust Supervised Fine-tuning for Large Language Models under
  Noisy Response
Authors: name: Junyu Luo, name: Xiao Luo, name: Kaize Ding, name: Jingyang Yuan, name: Zhiping Xiao, name: Ming Zhang
Published: 2024-12-19 15:00:18

Supervised fine-tuning (SFT) plays a crucial role in adapting large language
models (LLMs) to specific domains or tasks. However, as demonstrated by
empirical experiments, the collected data inevitably contains noise in
practical applications, which poses significant challenges to model performance
on downstream tasks. Therefore, there is an urgent need for a noise-robust SFT
framework to enhance model capabilities in downstream tasks. To address this
challenge, we introduce a robust SFT framework (RobustFT) that performs noise
detection and relabeling on downstream task data. For noise identification, our
approach employs a multi-expert collaborative system with inference-enhanced
models to achieve superior noise detection. In the denoising phase, we utilize
a context-enhanced strategy, which incorporates the most relevant and confident
knowledge followed by careful assessment to generate reliable annotations.
Additionally, we introduce an effective data selection mechanism based on
response entropy, ensuring only high-quality samples are retained for
fine-tuning. Extensive experiments conducted on multiple LLMs across five
datasets demonstrate RobustFT's exceptional performance in noisy scenarios.
Revisiting In-Context Learning with Long Context Language Models
Authors: name: Jinheon Baek, name: Sun Jae Lee, name: Prakhar Gupta, name: Geunseob, name: Oh, name: Siddharth Dalmia, name: Prateek Kolhar
Published: 2024-12-22 08:55:19

In-Context Learning (ICL) is a technique by which language models make
predictions based on examples provided in their input context. Previously,
their context window size imposed a limit on the number of examples that can be
shown, making example selection techniques crucial for identifying the
maximally effective set of examples. However, the recent advent of Long Context
Language Models (LCLMs) has significantly increased the number of examples that
can be included in context, raising an important question of whether ICL
performance in a many-shot regime is still sensitive to the method of sample
selection. To answer this, we revisit these approaches in the context of LCLMs
through extensive experiments on 18 datasets spanning 4 tasks. Surprisingly, we
observe that sophisticated example selection techniques do not yield
significant improvements over a simple random sample selection method. Instead,
we find that the advent of LCLMs has fundamentally shifted the challenge of ICL
from that of selecting the most effective examples to that of collecting
sufficient examples to fill the context window. Specifically, in certain
datasets, including all available examples does not fully utilize the context
window; however, by augmenting the examples in context with a simple data
augmentation approach, we substantially improve ICL performance by 5%.
Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and
  Dual Evaluation Metrics for Factuality and Coverage
Authors: name: Saehyung Lee, name: Seunghyun Yoon, name: Trung Bui, name: Jing Shi, name: Sungroh Yoon
Published: 2024-12-20 01:37:22

Multimodal large language models (MLLMs) excel at generating highly detailed
captions but often produce hallucinations. Our analysis reveals that existing
hallucination detection methods struggle with detailed captions. We attribute
this to the increasing reliance of MLLMs on their generated text, rather than
the input image, as the sequence length grows. To address this issue, we
propose a multiagent approach that leverages LLM-MLLM collaboration to correct
given captions. Additionally, we introduce an evaluation framework and a
benchmark dataset to facilitate the systematic analysis of detailed captions.
Our experiments demonstrate that our proposed evaluation method better aligns
with human judgments of factuality than existing metrics and that existing
approaches to improve the MLLM factuality may fall short in hyper-detailed
image captioning tasks. In contrast, our proposed method significantly enhances
the factual accuracy of captions, even improving those generated by GPT-4V.
Finally, we highlight a limitation of VQA-centric benchmarking by demonstrating
that an MLLM's performance on VQA benchmarks may not correlate with its ability
to generate detailed image captions.
TRecViT: A Recurrent Video Transformer
Authors: name: Viorica Pătrăucean, name: Xu Owen He, name: Joseph Heyward, name: Chuhan Zhang, name: Mehdi S. M. Sajjadi, name: George-Cristian Muraru, name: Artem Zholus, name: Mahdi Karami, name: Ross Goroshin, name: Yutian Chen, name: Simon Osindero, name: João Carreira, name: Razvan Pascanu
Published: 2024-12-18 19:44:30

We propose a novel block for video modelling. It relies on a
time-space-channel factorisation with dedicated blocks for each dimension:
gated linear recurrent units (LRUs) perform information mixing over time,
self-attention layers perform mixing over space, and MLPs over channels. The
resulting architecture TRecViT performs well on sparse and dense tasks, trained
in supervised or self-supervised regimes. Notably, our model is causal and
outperforms or is on par with a pure attention model ViViT-L on large scale
video datasets (SSv2, Kinetics400), while having 3times less parameters,
12times smaller memory footprint, and 5times lower FLOPs count. Code and
checkpoints will be made available online at
https://github.com/google-deepmind/trecvit.
Multi-LLM Text Summarization
Authors: name: Jiangnan Fang, name: Cheng-Tse Liu, name: Jieun Kim, name: Yash Bhedaru, name: Ethan Liu, name: Nikhil Singh, name: Nedim Lipka, name: Puneet Mathur, name: Nesreen K. Ahmed, name: Franck Dernoncourt, name: Ryan A. Rossi, name: Hanieh Deilamsalehy
Published: 2024-12-20 01:55:26

In this work, we propose a Multi-LLM summarization framework, and investigate
two different multi-LLM strategies including centralized and decentralized. Our
multi-LLM summarization framework has two fundamentally important steps at each
round of conversation: generation and evaluation. These steps are different
depending on whether our multi-LLM decentralized summarization is used or
centralized. In both our multi-LLM decentralized and centralized strategies, we
have k different LLMs that generate diverse summaries of the text. However,
during evaluation, our multi-LLM centralized summarization approach leverages a
single LLM to evaluate the summaries and select the best one whereas k LLMs are
used for decentralized multi-LLM summarization. Overall, we find that our
multi-LLM summarization approaches significantly outperform the baselines that
leverage only a single LLM by up to 3x. These results indicate the
effectiveness of multi-LLM approaches for summarization.
IDOL: Instant Photorealistic 3D Human Creation from a Single Image
Authors: name: Yiyu Zhuang, name: Jiaxi Lv, name: Hao Wen, name: Qing Shuai, name: Ailing Zeng, name: Hao Zhu, name: Shifeng Chen, name: Yujiu Yang, name: Xun Cao, name: Wei Liu
Published: 2024-12-19 15:43:05

Creating a high-fidelity, animatable 3D full-body avatar from a single image
is a challenging task due to the diverse appearance and poses of humans and the
limited availability of high-quality training data. To achieve fast and
high-quality human reconstruction, this work rethinks the task from the
perspectives of dataset, model, and representation. First, we introduce a
large-scale HUman-centric GEnerated dataset, HuGe100K, consisting of 100K
diverse, photorealistic sets of human images. Each set contains 24-view frames
in specific human poses, generated using a pose-controllable
image-to-multi-view model. Next, leveraging the diversity in views, poses, and
appearances within HuGe100K, we develop a scalable feed-forward transformer
model to predict a 3D human Gaussian representation in a uniform space from a
given human image. This model is trained to disentangle human pose, body shape,
clothing geometry, and texture. The estimated Gaussians can be animated without
post-processing. We conduct comprehensive experiments to validate the
effectiveness of the proposed dataset and method. Our model demonstrates the
ability to efficiently reconstruct photorealistic humans at 1K resolution from
a single input image using a single GPU instantly. Additionally, it seamlessly
supports various applications, as well as shape and texture editing tasks.
LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Gaps
Authors: name: Felix Friedrich, name: Simone Tedeschi, name: Patrick Schramowski, name: Manuel Brack, name: Roberto Navigli, name: Huu Nguyen, name: Bo Li, name: Kristian Kersting
Published: 2024-12-19 16:46:54

Building safe Large Language Models (LLMs) across multiple languages is
essential in ensuring both safe access and linguistic diversity. To this end,
we introduce M-ALERT, a multilingual benchmark that evaluates the safety of
LLMs in five languages: English, French, German, Italian, and Spanish. M-ALERT
includes 15k high-quality prompts per language, totaling 75k, following the
detailed ALERT taxonomy. Our extensive experiments on 10 state-of-the-art LLMs
highlight the importance of language-specific safety analysis, revealing that
models often exhibit significant inconsistencies in safety across languages and
categories. For instance, Llama3.2 shows high unsafety in the category
crime_tax for Italian but remains safe in other languages. Similar differences
can be observed across all models. In contrast, certain categories, such as
substance_cannabis and crime_propaganda, consistently trigger unsafe responses
across models and languages. These findings underscore the need for robust
multilingual safety practices in LLMs to ensure safe and responsible usage
across diverse user communities.
Sequence Matters: Harnessing Video Models in 3D Super-Resolution
Authors: name: Hyun-kyu Ko, name: Dongheok Park, name: Youngin Park, name: Byeonghyeon Lee, name: Juhee Han, name: Eunbyung Park
Published: 2024-12-16 08:00:50

3D super-resolution aims to reconstruct high-fidelity 3D models from
low-resolution (LR) multi-view images. Early studies primarily focused on
single-image super-resolution (SISR) models to upsample LR images into
high-resolution images. However, these methods often lack view consistency
because they operate independently on each image. Although various
post-processing techniques have been extensively explored to mitigate these
inconsistencies, they have yet to fully resolve the issues. In this paper, we
perform a comprehensive study of 3D super-resolution by leveraging video
super-resolution (VSR) models. By utilizing VSR models, we ensure a higher
degree of spatial consistency and can reference surrounding spatial
information, leading to more accurate and detailed reconstructions. Our
findings reveal that VSR models can perform remarkably well even on sequences
that lack precise spatial alignment. Given this observation, we propose a
simple yet practical approach to align LR images without involving fine-tuning
or generating 'smooth' trajectory from the trained 3D models over LR images.
The experimental results show that the surprisingly simple algorithms can
achieve the state-of-the-art results of 3D super-resolution tasks on standard
benchmark datasets, such as the NeRF-synthetic and MipNeRF-360 datasets.
Project page: https://ko-lani.github.io/Sequence-Matters
Fietje: An open, efficient LLM for Dutch
Authors: name: Bram Vanroy
Published: 2024-12-19 23:06:01

This paper introduces Fietje, a family of small language models (SLMs)
specifically designed for the Dutch language. The model is based on Phi 2, an
English-centric model of 2.7 billion parameters. Fietje demonstrated
competitive results with larger language models upon its release. A core
emphasis of this work is transparency and reproducibility: Fietje is fully
open-source, with model weights, datasets, training, and evaluation code all
publicly accessible.
  The paper discusses the performance of Fietje and many other models on an
extensive evaluation suite of benchmarks on reasoning, sentiment analysis,
world knowledge, linguistic acceptability and word sense disambiguation.
Evaluation results illustrate the rapid progress in the field of LLMs, where
recent small models outperform older, larger models that were fine-tuned for
Dutch. This trend signals an exciting future for Dutch language processing,
suggesting that even compact LLMs are becoming increasingly capable.
Furthermore, ongoing and future efforts to adapt LLMs to Dutch are poised to
enhance these models even further, broadening their applicability and
accessibility. Fietje is only an intermediate step in improving accessibility
to language technology for users of the Dutch language.
Offline Reinforcement Learning for LLM Multi-Step Reasoning
Authors: name: Huaijie Wang, name: Shibo Hao, name: Hanze Dong, name: Shenao Zhang, name: Yilin Bao, name: Ziran Yang, name: Yi Wu
Published: 2024-12-20 18:49:45

Improving the multi-step reasoning ability of large language models (LLMs)
with offline reinforcement learning (RL) is essential for quickly adapting them
to complex tasks. While Direct Preference Optimization (DPO) has shown promise
in aligning LLMs with human preferences, it is less suitable for multi-step
reasoning tasks because (1) DPO relies on paired preference data, which is not
readily available for multi-step reasoning tasks, and (2) it treats all tokens
uniformly, making it ineffective for credit assignment in multi-step reasoning
tasks, which often come with sparse reward. In this work, we propose OREO
(Offline Reasoning Optimization), an offline RL method for enhancing LLM
multi-step reasoning. Building on insights from previous works of maximum
entropy reinforcement learning, it jointly learns a policy model and value
function by optimizing the soft Bellman Equation. We show in principle that it
reduces the need to collect pairwise data and enables better credit assignment.
Empirically, OREO surpasses existing offline learning methods on multi-step
reasoning benchmarks, including mathematical reasoning tasks (GSM8K, MATH) and
embodied agent control (ALFWorld). The approach can be extended to a
multi-iteration framework when additional resources are available. Furthermore,
the learned value function can be leveraged to guide the tree search for free,
which can further boost performance during test time.
MixLLM: LLM Quantization with Global Mixed-precision between
  Output-features and Highly-efficient System Design
Authors: name: Zhen Zheng, name: Xiaonan Song, name: Chuanjie Liu
Published: 2024-12-19 07:15:15

Quantization has become one of the most effective methodologies to compress
LLMs into smaller size. However, the existing quantization solutions still show
limitations of either non-negligible accuracy drop or system inefficiency. In
this paper, we make a comprehensive analysis of the general quantization
principles on their effect to the triangle of accuracy, memory consumption and
system efficiency. We propose MixLLM that explores the new optimization space
of mixed-precision quantization between output features based on the insight
that different output features matter differently in the model. MixLLM
identifies the output features with high salience in the global view rather
than within each single layer, effectively assigning the larger bit-width to
output features that need it most to achieve good accuracy with low memory
consumption. We present the sweet spot of quantization configuration of
algorithm-system co-design that leads to high accuracy and system efficiency.
To address the system challenge, we design the two-step dequantization to make
use of the int8 Tensor Core easily and fast data type conversion to reduce
dequantization overhead significantly, and present the software pipeline to
overlap the memory access, dequantization and the MatMul to the best. Extensive
experiments show that with only 10% more bits, the PPL increasement can be
reduced from about 0.5 in SOTA to within 0.2 for Llama 3.1 70B, while on
average MMLU-Pro improves by 0.93 over the SOTA of three popular models. In
addition to its superior accuracy, MixLLM also achieves state-of-the-art system
efficiency.
CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers
  Up
Authors: name: Songhua Liu, name: Zhenxiong Tan, name: Xinchao Wang
Published: 2024-12-20 17:57:09

Diffusion Transformers (DiT) have become a leading architecture in image
generation. However, the quadratic complexity of attention mechanisms, which
are responsible for modeling token-wise relationships, results in significant
latency when generating high-resolution images. To address this issue, we aim
at a linear attention mechanism in this paper that reduces the complexity of
pre-trained DiTs to linear. We begin our exploration with a comprehensive
summary of existing efficient attention mechanisms and identify four key
factors crucial for successful linearization of pre-trained DiTs: locality,
formulation consistency, high-rank attention maps, and feature integrity. Based
on these insights, we introduce a convolution-like local attention strategy
termed CLEAR, which limits feature interactions to a local window around each
query token, and thus achieves linear complexity. Our experiments indicate
that, by fine-tuning the attention layer on merely 10K self-generated samples
for 10K iterations, we can effectively transfer knowledge from a pre-trained
DiT to a student model with linear complexity, yielding results comparable to
the teacher model. Simultaneously, it reduces attention computations by 99.5%
and accelerates generation by 6.3 times for generating 8K-resolution images.
Furthermore, we investigate favorable properties in the distilled attention
layers, such as zero-shot generalization cross various models and plugins, and
improved support for multi-GPU parallel inference. Models and codes are
available here: https://github.com/Huage001/CLEAR.
Parallelized Autoregressive Visual Generation
Authors: name: Yuqing Wang, name: Shuhuai Ren, name: Zhijie Lin, name: Yujin Han, name: Haoyuan Guo, name: Zhenheng Yang, name: Difan Zou, name: Jiashi Feng, name: Xihui Liu
Published: 2024-12-19 17:59:54

Autoregressive models have emerged as a powerful approach for visual
generation but suffer from slow inference speed due to their sequential
token-by-token prediction process. In this paper, we propose a simple yet
effective approach for parallelized autoregressive visual generation that
improves generation efficiency while preserving the advantages of
autoregressive modeling. Our key insight is that parallel generation depends on
visual token dependencies-tokens with weak dependencies can be generated in
parallel, while strongly dependent adjacent tokens are difficult to generate
together, as their independent sampling may lead to inconsistencies. Based on
this observation, we develop a parallel generation strategy that generates
distant tokens with weak dependencies in parallel while maintaining sequential
generation for strongly dependent local tokens. Our approach can be seamlessly
integrated into standard autoregressive models without modifying the
architecture or tokenizer. Experiments on ImageNet and UCF-101 demonstrate that
our method achieves a 3.6x speedup with comparable quality and up to 9.5x
speedup with minimal quality degradation across both image and video generation
tasks. We hope this work will inspire future research in efficient visual
generation and unified autoregressive modeling. Project page:
https://epiphqny.github.io/PAR-project.
Taming Multimodal Joint Training for High-Quality Video-to-Audio
  Synthesis
Authors: name: Ho Kei Cheng, name: Masato Ishii, name: Akio Hayakawa, name: Takashi Shibuya, name: Alexander Schwing, name: Yuki Mitsufuji
Published: 2024-12-19 18:59:55

We propose to synthesize high-quality and synchronized audio, given video and
optional text conditions, using a novel multimodal joint training framework
MMAudio. In contrast to single-modality training conditioned on (limited) video
data only, MMAudio is jointly trained with larger-scale, readily available
text-audio data to learn to generate semantically aligned high-quality audio
samples. Additionally, we improve audio-visual synchrony with a conditional
synchronization module that aligns video conditions with audio latents at the
frame level. Trained with a flow matching objective, MMAudio achieves new
video-to-audio state-of-the-art among public models in terms of audio quality,
semantic alignment, and audio-visual synchronization, while having a low
inference time (1.23s to generate an 8s clip) and just 157M parameters. MMAudio
also achieves surprisingly competitive performance in text-to-audio generation,
showing that joint training does not hinder single-modality performance. Code
and demo are available at: https://hkchengrex.github.io/MMAudio
SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation
Authors: name: Jialong Wu, name: Zhenglin Wang, name: Linhai Zhang, name: Yilong Lai, name: Yulan He, name: Deyu Zhou
Published: 2024-12-18 09:27:33

Key-Value (KV) cache has become a bottleneck of LLMs for long-context
generation. Despite the numerous efforts in this area, the optimization for the
decoding phase is generally ignored. However, we believe such optimization is
crucial, especially for long-output generation tasks based on the following two
observations: (i) Excessive compression during the prefill phase, which
requires specific full context impairs the comprehension of the reasoning task;
(ii) Deviation of heavy hitters occurs in the reasoning tasks with long
outputs. Therefore, SCOPE, a simple yet efficient framework that separately
performs KV cache optimization during the prefill and decoding phases, is
introduced. Specifically, the KV cache during the prefill phase is preserved to
maintain the essential information, while a novel strategy based on sliding is
proposed to select essential heavy hitters for the decoding phase. Memory usage
and memory transfer are further optimized using adaptive and discontinuous
strategies. Extensive experiments on LongGenBench show the effectiveness and
generalization of SCOPE and its compatibility as a plug-in to other
prefill-only KV compression methods.