Orionnnn/hub-stats huggingface dataset:
EasyAnimate: A High-Performance Long Video Generation Method based on
  Transformer Architecture
Authors: name: Jiaqi Xu, name: Xinyi Zou, name: Kunzhe Huang, name: Yunkuo Chen, name: Bo Liu, name: MengLi Cheng, name: Xing Shi, name: Jun Huang
Published: 2024-05-29 11:11:07

This paper presents EasyAnimate, an advanced method for video generation that
leverages the power of transformer architecture for high-performance outcomes.
We have expanded the DiT framework originally designed for 2D image synthesis
to accommodate the complexities of 3D video generation by incorporating a
motion module block. It is used to capture temporal dynamics, thereby ensuring
the production of consistent frames and seamless motion transitions. The motion
module can be adapted to various DiT baseline methods to generate video with
different styles. It can also generate videos with different frame rates and
resolutions during both training and inference phases, suitable for both images
and videos. Moreover, we introduce slice VAE, a novel approach to condense the
temporal axis, facilitating the generation of long duration videos. Currently,
EasyAnimate exhibits the proficiency to generate videos with 144 frames. We
provide a holistic ecosystem for video production based on DiT, encompassing
aspects such as data pre-processing, VAE training, DiT models training (both
the baseline model and LoRA model), and end-to-end video inference. Code is
available at: https://github.com/aigc-apps/EasyAnimate. We are continuously
working to enhance the performance of our method.
T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model
  with Mixed Reward Feedback
Authors: name: Jiachen Li, name: Weixi Feng, name: Tsu-Jui Fu, name: Xinyi Wang, name: Sugato Basu, name: Wenhu Chen, name: William Yang Wang
Published: 2024-05-29 04:26:17

Diffusion-based text-to-video (T2V) models have achieved significant success
but continue to be hampered by the slow sampling speed of their iterative
sampling processes. To address the challenge, consistency models have been
proposed to facilitate fast inference, albeit at the cost of sample quality. In
this work, we aim to break the quality bottleneck of a video consistency model
(VCM) to achieve both fast and high-quality video generation. We
introduce T2V-Turbo, which integrates feedback from a mixture of differentiable
reward models into the consistency distillation (CD) process of a pre-trained
T2V model. Notably, we directly optimize rewards associated with single-step
generations that arise naturally from computing the CD loss, effectively
bypassing the memory constraints imposed by backpropagating gradients through
an iterative sampling process. Remarkably, the 4-step generations from our
T2V-Turbo achieve the highest total score on VBench, even surpassing Gen-2 and
Pika. We further conduct human evaluations to corroborate the results,
validating that the 4-step generations from our T2V-Turbo are preferred over
the 50-step DDIM samples from their teacher models, representing more than a
tenfold acceleration while improving video generation quality.
Offline Regularised Reinforcement Learning for Large Language Models
  Alignment
Authors: name: Pierre Harvey Richemond, name: Yunhao Tang, name: Daniel Guo, name: Daniele Calandriello, name: Mohammad Gheshlaghi Azar, name: Rafael Rafailov, name: Bernardo Avila Pires, name: Eugene Tarassov, name: Lucas Spangher, name: Will Ellsworth, name: Aliaksei Severyn, name: Jonathan Mallinson, name: Lior Shani, name: Gil Shamir, name: Rishabh Joshi, name: Tianqi Liu, name: Remi Munos, name: Bilal Piot
Published: 2024-05-29 14:11:29

The dominant framework for alignment of large language models (LLM), whether
through reinforcement learning from human feedback or direct preference
optimisation, is to learn from preference data. This involves building datasets
where each element is a quadruplet composed of a prompt, two independent
responses (completions of the prompt) and a human preference between the two
independent responses, yielding a preferred and a dis-preferred response. Such
data is typically scarce and expensive to collect. On the other hand,
single-trajectory datasets where each element is a triplet composed of a
prompt, a response and a human feedback is naturally more abundant. The
canonical element of such datasets is for instance an LLM's response to a
user's prompt followed by a user's feedback such as a thumbs-up/down.
Consequently, in this work, we propose DRO, or Direct Reward
Optimisation, as a framework and associated algorithms that do not require
pairwise preferences. DRO uses a simple mean-squared objective that can be
implemented in various ways. We validate our findings empirically, using T5
encoder-decoder language models, and show DRO's performance over selected
baselines such as Kahneman-Tversky Optimization (KTO). Thus, we confirm that
DRO is a simple and empirically compelling method for single-trajectory policy
optimisation.
Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities
Authors: name: Vicky Zayats, name: Peter Chen, name: Melissa Merrari, name: Dirk Padfield
Published: 2024-05-29 00:23:55

Integrating multiple generative foundation models, especially those trained
on different modalities, into something greater than the sum of its parts poses
significant challenges. Two key hurdles are the availability of aligned data
(concepts that contain similar meaning but is expressed differently in
different modalities), and effectively leveraging unimodal representations in
cross-domain generative tasks, without compromising their original unimodal
capabilities.
  We propose Zipper, a multi-tower decoder architecture that addresses these
concerns by using cross-attention to flexibly compose multimodal generative
models from independently pre-trained unimodal decoders. In our experiments
fusing speech and text modalities, we show the proposed architecture performs
very competitively in scenarios with limited aligned text-speech data. We also
showcase the flexibility of our model to selectively maintain unimodal (e.g.,
text-to-text generation) generation performance by freezing the corresponding
modal tower (e.g. text). In cross-modal tasks such as automatic speech
recognition (ASR) where the output modality is text, we show that freezing the
text backbone results in negligible performance degradation. In cross-modal
tasks such as text-to-speech generation (TTS) where the output modality is
speech, we show that using a pre-trained speech backbone results in superior
performance to the baseline.
Nearest Neighbor Speculative Decoding for LLM Generation and Attribution
Authors: name: Minghan Li, name: Xilun Chen, name: Ari Holtzman, name: Beidi Chen, name: Jimmy Lin, name: Wen-tau Yih, name: Xi Victoria Lin
Published: 2024-05-29 17:55:03

Large language models (LLMs) often hallucinate and lack the ability to
provide attribution for their generations. Semi-parametric LMs, such as kNN-LM,
approach these limitations by refining the output of an LM for a given prompt
using its nearest neighbor matches in a non-parametric data store. However,
these models often exhibit slow inference speeds and produce non-fluent texts.
In this paper, we introduce Nearest Neighbor Speculative Decoding (NEST), a
novel semi-parametric language modeling approach that is capable of
incorporating real-world text spans of arbitrary length into the LM generations
and providing attribution to their sources. NEST performs token-level retrieval
at each inference step to compute a semi-parametric mixture distribution and
identify promising span continuations in a corpus. It then uses an approximate
speculative decoding procedure that accepts a prefix of the retrieved span or
generates a new token. NEST significantly enhances the generation quality and
attribution rate of the base LM across a variety of knowledge-intensive tasks,
surpassing the conventional kNN-LM method and performing competitively with
in-context retrieval augmentation. In addition, NEST substantially improves the
generation speed, achieving a 1.8x speedup in inference time when applied to
Llama-2-Chat 70B.
Value-Incentivized Preference Optimization: A Unified Approach to Online
  and Offline RLHF
Authors: name: Shicong Cen, name: Jincheng Mei, name: Katayoon Goshvadi, name: Hanjun Dai, name: Tong Yang, name: Sherry Yang, name: Dale Schuurmans, name: Yuejie Chi, name: Bo Dai
Published: 2024-05-29 17:51:42

Reinforcement learning from human feedback (RLHF) has demonstrated great
promise in aligning large language models (LLMs) with human preference.
Depending on the availability of preference data, both online and offline RLHF
are active areas of investigation. A key bottleneck is understanding how to
incorporate uncertainty estimation in the reward function learned from the
preference data for RLHF, regardless of how the preference data is collected.
While the principles of optimism or pessimism under uncertainty are
well-established in standard reinforcement learning (RL), a
practically-implementable and theoretically-grounded form amenable to large
language models is not yet available, as standard techniques for constructing
confidence intervals become intractable under arbitrary policy
parameterizations.
  In this paper, we introduce a unified approach to online and offline RLHF --
value-incentivized preference optimization (VPO) -- which regularizes the
maximum-likelihood estimate of the reward function with the corresponding value
function, modulated by a sign to indicate whether the optimism or
pessimism is chosen. VPO also directly optimizes the policy with implicit
reward modeling, and therefore shares a simpler RLHF pipeline similar to direct
preference optimization. Theoretical guarantees of VPO are provided for both
online and offline settings, matching the rates of their standard RL
counterparts. Moreover, experiments on text summarization and dialog verify the
practicality and effectiveness of VPO.
Self-Exploring Language Models: Active Preference Elicitation for Online
  Alignment
Authors: name: Shenao Zhang, name: Donghan Yu, name: Hiteshi Sharma, name: Ziyi Yang, name: Shuohang Wang, name: Hany Hassan, name: Zhaoran Wang
Published: 2024-05-29 17:59:07

Preference optimization, particularly through Reinforcement Learning from
Human Feedback (RLHF), has achieved significant success in aligning Large
Language Models (LLMs) to adhere to human intentions. Unlike offline alignment
with a fixed dataset, online feedback collection from humans or AI on model
generations typically leads to more capable reward models and better-aligned
LLMs through an iterative process. However, achieving a globally accurate
reward model requires systematic exploration to generate diverse responses that
span the vast space of natural language. Random sampling from standard
reward-maximizing LLMs alone is insufficient to fulfill this requirement. To
address this issue, we propose a bilevel objective optimistically biased
towards potentially high-reward responses to actively explore
out-of-distribution regions. By solving the inner-level problem with the
reparameterized reward function, the resulting algorithm, named Self-Exploring
Language Models (SELM), eliminates the need for a separate RM and iteratively
updates the LLM with a straightforward objective. Compared to Direct Preference
Optimization (DPO), the SELM objective reduces indiscriminate favor of unseen
extrapolations and enhances exploration efficiency. Our experimental results
demonstrate that when finetuned on Zephyr-7B-SFT and Llama-3-8B-Instruct
models, SELM significantly boosts the performance on instruction-following
benchmarks such as MT-Bench and AlpacaEval 2.0, as well as various standard
academic benchmarks in different settings. Our code and models are available at
https://github.com/shenao-zhang/SELM.
MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model
  Series
Authors: name: Ge Zhang, name: Scott Qu, name: Jiaheng Liu, name: Chenchen Zhang, name: Chenghua Lin, name: Chou Leuang Yu, name: Danny Pan, name: Esther Cheng, name: Jie Liu, name: Qunshu Lin, name: Raven Yuan, name: Tuney Zheng, name: Wei Pang, name: Xinrun Du, name: Yiming Liang, name: Yinghao Ma, name: Yizhi Li, name: Ziyang Ma, name: Bill Lin, name: Emmanouil Benetos, name: Huan Yang, name: Junting Zhou, name: Kaijing Ma, name: Minghao Liu, name: Morry Niu, name: Noah Wang, name: Quehry Que, name: Ruibo Liu, name: Sine Liu, name: Shawn Guo, name: Soren Gao, name: Wangchunshu Zhou, name: Xinyue Zhang, name: Yizhi Zhou, name: Yubo Wang, name: Yuelin Bai, name: Yuhan Zhang, name: Yuxiang Zhang, name: Zenith Wang, name: Zhenzhu Yang, name: Zijian Zhao, name: Jiajun Zhang, name: Wanli Ouyang, name: Wenhao Huang, name: Wenhu Chen
Published: 2024-05-29 17:57:16

Large Language Models (LLMs) have made great strides in recent years to
achieve unprecedented performance across different tasks. However, due to
commercial interest, the most competitive models like GPT, Gemini, and Claude
have been gated behind proprietary interfaces without disclosing the training
details. Recently, many institutions have open-sourced several strong LLMs like
LLaMA-3, comparable to existing closed-source LLMs. However, only the model's
weights are provided with most details (e.g., intermediate checkpoints,
pre-training corpus, and training code, etc.) being undisclosed. To improve the
transparency of LLMs, the research community has formed to open-source truly
open LLMs (e.g., Pythia, Amber, OLMo), where more details (e.g., pre-training
corpus and training code) are being provided. These models have greatly
advanced the scientific study of these large models including their strengths,
weaknesses, biases and risks. However, we observe that the existing truly open
LLMs on reasoning, knowledge, and coding tasks are still inferior to existing
state-of-the-art LLMs with similar model sizes. To this end, we open-source
MAP-Neo, a highly capable and transparent bilingual language model with 7B
parameters trained from scratch on 4.5T high-quality tokens. Our MAP-Neo is the
first fully open-sourced bilingual LLM with comparable performance compared to
existing state-of-the-art LLMs. Moreover, we open-source all details to
reproduce our MAP-Neo, where the cleaned pre-training corpus, data cleaning
pipeline, checkpoints, and well-optimized training/evaluation framework are
provided. Finally, we hope our MAP-Neo will enhance and strengthen the open
research community and inspire more innovations and creativities to facilitate
the further improvements of LLMs.
3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian
  Splatting
Authors: name: Qihang Zhang, name: Yinghao Xu, name: Chaoyang Wang, name: Hsin-Ying Lee, name: Gordon Wetzstein, name: Bolei Zhou, name: Ceyuan Yang
Published: 2024-05-28 17:59:01

Scene image editing is crucial for entertainment, photography, and
advertising design. Existing methods solely focus on either 2D individual
object or 3D global scene editing. This results in a lack of a unified approach
to effectively control and manipulate scenes at the 3D level with different
levels of granularity. In this work, we propose 3DitScene, a novel and unified
scene editing framework leveraging language-guided disentangled Gaussian
Splatting that enables seamless editing from 2D to 3D, allowing precise control
over scene composition and individual objects. We first incorporate 3D
Gaussians that are refined through generative priors and optimization
techniques. Language features from CLIP then introduce semantics into 3D
geometry for object disentanglement. With the disentangled Gaussians, 3DitScene
allows for manipulation at both the global and individual levels,
revolutionizing creative expression and empowering control over scenes and
objects. Experimental results demonstrate the effectiveness and versatility of
3DitScene in scene image editing. Code and online demo can be found at our
project homepage: https://zqh0253.github.io/3DitScene/.
GFlow: Recovering 4D World from Monocular Video
Authors: name: Shizun Wang, name: Xingyi Yang, name: Qiuhong Shen, name: Zhenxiang Jiang, name: Xinchao Wang
Published: 2024-05-28 17:59:22

Reconstructing 4D scenes from video inputs is a crucial yet challenging task.
Conventional methods usually rely on the assumptions of multi-view video
inputs, known camera parameters, or static scenes, all of which are typically
absent under in-the-wild scenarios. In this paper, we relax all these
constraints and tackle a highly ambitious but practical task, which we termed
as AnyV4D: we assume only one monocular video is available without any camera
parameters as input, and we aim to recover the dynamic 4D world alongside the
camera poses. To this end, we introduce GFlow, a new framework that utilizes
only 2D priors (depth and optical flow) to lift a video (3D) to a 4D explicit
representation, entailing a flow of Gaussian splatting through space and time.
GFlow first clusters the scene into still and moving parts, then applies a
sequential optimization process that optimizes camera poses and the dynamics of
3D Gaussian points based on 2D priors and scene clustering, ensuring fidelity
among neighboring points and smooth movement across frames. Since dynamic
scenes always introduce new content, we also propose a new pixel-wise
densification strategy for Gaussian points to integrate new visual content.
Moreover, GFlow transcends the boundaries of mere 4D reconstruction; it also
enables tracking of any points across frames without the need for prior
training and segments moving objects from the scene in an unsupervised way.
Additionally, the camera poses of each frame can be derived from GFlow,
allowing for rendering novel views of a video scene through changing camera
pose. By employing the explicit representation, we may readily conduct
scene-level or object-level editing as desired, underscoring its versatility
and power. Visit our project website at: https://littlepure2333.github.io/GFlow
Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language
  Models via Instruction Tuning
Authors: name: Yixiao Zhang, name: Yukara Ikemiya, name: Woosung Choi, name: Naoki Murata, name: Marco A. Martínez-Ramírez, name: Liwei Lin, name: Gus Xia, name: Wei-Hsiang Liao, name: Yuki Mitsufuji, name: Simon Dixon
Published: 2024-05-28 17:27:20

Recent advances in text-to-music editing, which employ text queries to modify
music (e.g.\ by changing its style or adjusting instrumental components),
present unique challenges and opportunities for AI-assisted music creation.
Previous approaches in this domain have been constrained by the necessity to
train specific editing models from scratch, which is both resource-intensive
and inefficient; other research uses large language models to predict edited
music, resulting in imprecise audio reconstruction. To Combine the strengths
and address these limitations, we introduce Instruct-MusicGen, a novel approach
that finetunes a pretrained MusicGen model to efficiently follow editing
instructions such as adding, removing, or separating stems. Our approach
involves a modification of the original MusicGen architecture by incorporating
a text fusion module and an audio fusion module, which allow the model to
process instruction texts and audio inputs concurrently and yield the desired
edited music. Remarkably, Instruct-MusicGen only introduces 8% new parameters
to the original MusicGen model and only trains for 5K steps, yet it achieves
superior performance across all tasks compared to existing baselines, and
demonstrates performance comparable to the models trained for specific tasks.
This advancement not only enhances the efficiency of text-to-music editing but
also broadens the applicability of music language models in dynamic music
production environments.
LLaMA-NAS: Efficient Neural Architecture Search for Large Language
  Models
Authors: name: Anthony Sarah, name: Sharath Nittur Sridhar, name: Maciej Szankin, name: Sairam Sundaresan
Published: 2024-05-28 17:20:44

The abilities of modern large language models (LLMs) in solving natural
language processing, complex reasoning, sentiment analysis and other tasks have
been extraordinary which has prompted their extensive adoption. Unfortunately,
these abilities come with very high memory and computational costs which
precludes the use of LLMs on most hardware platforms. To mitigate this, we
propose an effective method of finding Pareto-optimal network architectures
based on LLaMA2-7B using one-shot NAS. In particular, we fine-tune LLaMA2-7B
only once and then apply genetic algorithm-based search to find smaller, less
computationally complex network architectures. We show that, for certain
standard benchmark tasks, the pre-trained LLaMA2-7B network is unnecessarily
large and complex. More specifically, we demonstrate a 1.5x reduction in model
size and 1.3x speedup in throughput for certain tasks with negligible drop in
accuracy. In addition to finding smaller, higher-performing network
architectures, our method does so more effectively and efficiently than certain
pruning or sparsification techniques. Finally, we demonstrate how quantization
is complementary to our method and that the size and complexity of the networks
we find can be further decreased using quantization. We believe that our work
provides a way to automatically create LLMs which can be used on less expensive
and more readily available hardware platforms.
2BP: 2-Stage Backpropagation
Authors: name: Christopher Rae, name: Joseph K. L. Lee, name: James Richings
Published: 2024-05-28 11:02:01

As Deep Neural Networks (DNNs) grow in size and complexity, they often exceed
the memory capacity of a single accelerator, necessitating the sharding of
model parameters across multiple accelerators. Pipeline parallelism is a
commonly used sharding strategy for training large DNNs. However, current
implementations of pipeline parallelism are being unintentionally bottlenecked
by the automatic differentiation tools provided by ML frameworks. This paper
introduces 2-stage backpropagation (2BP). By splitting the backward propagation
step into two separate stages, we can reduce idle compute time. We tested 2BP
on various model architectures and pipelining schedules, achieving increases in
throughput in all cases. Using 2BP, we were able to achieve a 1.70x increase in
throughput compared to traditional methods when training a LLaMa-like
transformer with 7 billion parameters across 4 GPUs.
VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections
Authors: name: Roy Miles, name: Pradyumna Reddy, name: Ismail Elezi, name: Jiankang Deng
Published: 2024-05-28 09:23:14

Large language models (LLMs) have recently emerged as powerful tools for
tackling many language-processing tasks. Despite their success, training and
fine-tuning these models is still far too computationally and memory intensive.
In this paper, we identify and characterise the important components needed for
effective model convergence using gradient descent. In doing so we find that
the intermediate activations used to implement backpropagation can be
excessively compressed without incurring any degradation in performance. This
result leads us to a cheap and memory-efficient algorithm for both fine-tuning
and pre-training LLMs. The proposed algorithm simply divides the tokens up into
smaller sub-tokens before projecting them onto a fixed 1-dimensional subspace
during the forward pass. These features are then coarsely reconstructed during
the backward pass to implement the update rules. We confirm the effectiveness
of our algorithm as being complimentary to many state-of-the-art PEFT methods
on the VTAB-1k fine-tuning benchmark. Furthermore, we outperform QLoRA for
fine-tuning LLaMA and show competitive performance against other
memory-efficient pre-training methods on the large-scale C4 dataset.