Orionnnn/hub-stats huggingface dataset:
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of
  Language Models
Authors: name: Mert Yuksekgonul, name: Varun Chandrasekaran, name: Erik Jones, name: Suriya Gunasekar, name: Ranjita Naik, name: Hamid Palangi, name: Ece Kamar, name: Besmira Nushi
Published: 2023-09-26 17:48:55

We investigate the internal behavior of Transformer-based Large Language
Models (LLMs) when they generate factually incorrect text. We propose modeling
factual queries as Constraint Satisfaction Problems and use this framework to
investigate how the model interacts internally with factual constraints.
Specifically, we discover a strong positive relation between the model's
attention to constraint tokens and the factual accuracy of its responses. In
our curated suite of 11 datasets with over 40,000 prompts, we study the task of
predicting factual errors with the Llama-2 family across all scales (7B, 13B,
70B). We propose SAT Probe, a method probing self-attention patterns, that can
predict constraint satisfaction and factual errors, and allows early error
identification. The approach and findings demonstrate how using the mechanistic
understanding of factuality in LLMs can enhance reliability.
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models
Authors: name: Yuhui Xu, name: Lingxi Xie, name: Xiaotao Gu, name: Xin Chen, name: Heng Chang, name: Hengheng Zhang, name: Zhensu Chen, name: Xiaopeng Zhang, name: Qi Tian
Published: 2023-09-26 07:22:23

Recently years have witnessed a rapid development of large language models
(LLMs). Despite the strong ability in many language-understanding tasks, the
heavy computational burden largely restricts the application of LLMs especially
when one needs to deploy them onto edge devices. In this paper, we propose a
quantization-aware low-rank adaptation (QA-LoRA) algorithm. The motivation lies
in the imbalanced degrees of freedom of quantization and adaptation, and the
solution is to use group-wise operators which increase the degree of freedom of
quantization meanwhile decreasing that of adaptation. QA-LoRA is easily
implemented with a few lines of code, and it equips the original LoRA with
two-fold abilities: (i) during fine-tuning, the LLM's weights are quantized
(e.g., into INT4) to reduce time and memory usage; (ii) after fine-tuning, the
LLM and auxiliary weights are naturally integrated into a quantized model
without loss of accuracy. We apply QA-LoRA to the LLaMA and LLaMA2 model
families and validate its effectiveness in different fine-tuning datasets and
downstream scenarios. Code will be made available at
https://github.com/yuhuixu1993/qa-lora.
Efficient Post-training Quantization with FP8 Formats
Authors: name: Haihao Shen, name: Naveen Mellempudi, name: Xin He, name: Qun Gao, name: Chang Wang, name: Mengni Wang
Published: 2023-09-26 00:58:36

Recent advances in deep learning methods such as LLMs and Diffusion models
have created a need for improved quantization methods that can meet the
computational demands of these modern architectures while maintaining accuracy.
Towards this goal, we study the advantages of FP8 data formats for
post-training quantization across 75 unique network architectures covering a
wide range of tasks, including machine translation, language modeling, text
generation, image classification, generation, and segmentation. We examine
three different FP8 representations (E5M2, E4M3, and E3M4) to study the effects
of varying degrees of trade-off between dynamic range and precision on model
accuracy. Based on our extensive study, we developed a quantization workflow
that generalizes across different network architectures. Our empirical results
show that FP8 formats outperform INT8 in multiple aspects, including workload
coverage (92.64% vs. 65.87%), model accuracy and suitability for a broader
range of operations. Furthermore, our findings suggest that E4M3 is better
suited for NLP models, whereas E3M4 performs marginally better than E4M3 on
computer vision tasks. The code is publicly available on Intel Neural
Compressor: https://github.com/intel/neural-compressor.
DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme
  Long Sequence Transformer Models
Authors: name: Sam Ade Jacobs, name: Masahiro Tanaka, name: Chengming Zhang, name: Minjia Zhang, name: Leon Song, name: Samyam Rajbhandari, name: Yuxiong He
Published: 2023-09-25 20:15:57

Computation in a typical Transformer-based large language model (LLM) can be
characterized by batch size, hidden dimension, number of layers, and sequence
length. Until now, system works for accelerating LLM training have focused on
the first three dimensions: data parallelism for batch size, tensor parallelism
for hidden size and pipeline parallelism for model depth or layers. These
widely studied forms of parallelism are not targeted or optimized for long
sequence Transformer models. Given practical application needs for long
sequence LLM, renewed attentions are being drawn to sequence parallelism.
However, existing works in sequence parallelism are constrained by
memory-communication inefficiency, limiting their scalability to long sequence
large models. In this work, we introduce DeepSpeed-Ulysses, a novel, portable
and effective methodology for enabling highly efficient and scalable LLM
training with extremely long sequence length. DeepSpeed-Ulysses at its core
partitions input data along the sequence dimension and employs an efficient
all-to-all collective communication for attention computation. Theoretical
communication analysis shows that whereas other methods incur communication
overhead as sequence length increases, DeepSpeed-Ulysses maintains constant
communication volume when sequence length and compute devices are increased
proportionally. Furthermore, experimental evaluations show that
DeepSpeed-Ulysses trains 2.5X faster with 4X longer sequence length than the
existing method SOTA baseline.
LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion
  Models
Authors: name: Yaohui Wang, name: Xinyuan Chen, name: Xin Ma, name: Shangchen Zhou, name: Ziqi Huang, name: Yi Wang, name: Ceyuan Yang, name: Yinan He, name: Jiashuo Yu, name: Peiqing Yang, name: Yuwei Guo, name: Tianxing Wu, name: Chenyang Si, name: Yuming Jiang, name: Cunjian Chen, name: Chen Change Loy, name: Bo Dai, name: Dahua Lin, name: Yu Qiao, name: Ziwei Liu
Published: 2023-09-26 17:52:03

This work aims to learn a high-quality text-to-video (T2V) generative model
by leveraging a pre-trained text-to-image (T2I) model as a basis. It is a
highly desirable yet challenging task to simultaneously a) accomplish the
synthesis of visually realistic and temporally coherent videos while b)
preserving the strong creative generation nature of the pre-trained T2I model.
To this end, we propose LaVie, an integrated video generation framework that
operates on cascaded video latent diffusion models, comprising a base T2V
model, a temporal interpolation model, and a video super-resolution model. Our
key insights are two-fold: 1) We reveal that the incorporation of simple
temporal self-attentions, coupled with rotary positional encoding, adequately
captures the temporal correlations inherent in video data. 2) Additionally, we
validate that the process of joint image-video fine-tuning plays a pivotal role
in producing high-quality and creative outcomes. To enhance the performance of
LaVie, we contribute a comprehensive and diverse video dataset named Vimeo25M,
consisting of 25 million text-video pairs that prioritize quality, diversity,
and aesthetic appeal. Extensive experiments demonstrate that LaVie achieves
state-of-the-art performance both quantitatively and qualitatively.
Furthermore, we showcase the versatility of pre-trained LaVie models in various
long video generation and personalized video synthesis applications.
VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided
  Planning
Authors: name: Han Lin, name: Abhay Zala, name: Jaemin Cho, name: Mohit Bansal
Published: 2023-09-26 17:36:26

Although recent text-to-video (T2V) generation methods have seen significant
advancements, most of these works focus on producing short video clips of a
single event with a single background (i.e., single-scene videos). Meanwhile,
recent large language models (LLMs) have demonstrated their capability in
generating layouts and programs to control downstream visual modules such as
image generation models. This raises an important question: can we leverage the
knowledge embedded in these LLMs for temporally consistent long video
generation? In this paper, we propose VideoDirectorGPT, a novel framework for
consistent multi-scene video generation that uses the knowledge of LLMs for
video content planning and grounded video generation. Specifically, given a
single text prompt, we first ask our video planner LLM (GPT-4) to expand it
into a 'video plan', which involves generating the scene descriptions, the
entities with their respective layouts, the background for each scene, and
consistency groupings of the entities and backgrounds. Next, guided by this
output from the video planner, our video generator, Layout2Vid, has explicit
control over spatial layouts and can maintain temporal consistency of
entities/backgrounds across scenes, while only trained with image-level
annotations. Our experiments demonstrate that VideoDirectorGPT framework
substantially improves layout and movement control in both single- and
multi-scene video generation and can generate multi-scene videos with visual
consistency across scenes, while achieving competitive performance with SOTAs
in open-domain single-scene T2V generation. We also demonstrate that our
framework can dynamically control the strength for layout guidance and can also
generate videos with user-provided images. We hope our framework can inspire
future work on better integrating the planning ability of LLMs into consistent
long video generation.
Aligning Large Multimodal Models with Factually Augmented RLHF
Authors: name: Zhiqing Sun, name: Sheng Shen, name: Shengcao Cao, name: Haotian Liu, name: Chunyuan Li, name: Yikang Shen, name: Chuang Gan, name: Liang-Yan Gui, name: Yu-Xiong Wang, name: Yiming Yang, name: Kurt Keutzer, name: Trevor Darrell
Published: 2023-09-25 20:59:33

Large Multimodal Models (LMM) are built across modalities and the
misalignment between two modalities can result in "hallucination", generating
textual outputs that are not grounded by the multimodal information in context.
To address the multimodal misalignment issue, we adapt the Reinforcement
Learning from Human Feedback (RLHF) from the text domain to the task of
vision-language alignment, where human annotators are asked to compare two
responses and pinpoint the more hallucinated one, and the vision-language model
is trained to maximize the simulated human rewards. We propose a new alignment
algorithm called Factually Augmented RLHF that augments the reward model with
additional factual information such as image captions and ground-truth
multi-choice options, which alleviates the reward hacking phenomenon in RLHF
and further improves the performance. We also enhance the GPT-4-generated
training data (for vision instruction tuning) with previously available
human-written image-text pairs to improve the general capabilities of our
model. To evaluate the proposed approach in real-world scenarios, we develop a
new evaluation benchmark MMHAL-BENCH with a special focus on penalizing
hallucinations. As the first LMM trained with RLHF, our approach achieves
remarkable improvement on the LLaVA-Bench dataset with the 94% performance
level of the text-only GPT-4 (while previous best methods can only achieve the
87% level), and an improvement by 60% on MMHAL-BENCH over other baselines. We
opensource our code, model, data at https://llava-rlhf.github.io.
Small-scale proxies for large-scale Transformer training instabilities
Authors: name: Mitchell Wortsman, name: Peter J. Liu, name: Lechao Xiao, name: Katie Everett, name: Alex Alemi, name: Ben Adlam, name: John D. Co-Reyes, name: Izzeddin Gur, name: Abhishek Kumar, name: Roman Novak, name: Jeffrey Pennington, name: Jascha Sohl-dickstein, name: Kelvin Xu, name: Jaehoon Lee, name: Justin Gilmer, name: Simon Kornblith
Published: 2023-09-25 17:48:51

Teams that have trained large Transformer-based models have reported training
instabilities at large scale that did not appear when training with the same
hyperparameters at smaller scales. Although the causes of such instabilities
are of scientific interest, the amount of resources required to reproduce them
has made investigation difficult. In this work, we seek ways to reproduce and
study training stability and instability at smaller scales. First, we focus on
two sources of training instability described in previous work: the growth of
logits in attention layers (Dehghani et al., 2023) and divergence of the output
logits from the log probabilities (Chowdhery et al., 2022). By measuring the
relationship between learning rate and loss across scales, we show that these
instabilities also appear in small models when training at high learning rates,
and that mitigations previously employed at large scales are equally effective
in this regime. This prompts us to investigate the extent to which other known
optimizer and model interventions influence the sensitivity of the final loss
to changes in the learning rate. To this end, we study methods such as warm-up,
weight decay, and the muParam (Yang et al., 2022), and combine techniques to
train small models that achieve similar losses across orders of magnitude of
learning rate variation. Finally, to conclude our exploration we study two
cases where instabilities can be predicted before they emerge by examining the
scaling behavior of model activation and gradient norms.
DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via
  Multi-Modal Causal Attention
Authors: name: Zhewei Yao, name: Xiaoxia Wu, name: Conglong Li, name: Minjia Zhang, name: Heyang Qi, name: Olatunji Ruwase, name: Ammar Ahmad Awan, name: Samyam Rajbhandari, name: Yuxiong He
Published: 2023-09-25 17:53:29

Most of the existing multi-modal models, hindered by their incapacity to
adeptly manage interleaved image-and-text inputs in multi-image, multi-round
dialogues, face substantial constraints in resource allocation for training and
data accessibility, impacting their adaptability and scalability across varied
interaction realms. To address this, we present the DeepSpeed-VisualChat
framework, designed to optimize Large Language Models (LLMs) by incorporating
multi-modal capabilities, with a focus on enhancing the proficiency of Large
Vision and Language Models in handling interleaved inputs. Our framework is
notable for (1) its open-source support for multi-round and multi-image
dialogues, (2) introducing an innovative multi-modal causal attention
mechanism, and (3) utilizing data blending techniques on existing datasets to
assure seamless interactions in multi-round, multi-image conversations.
Compared to existing frameworks, DeepSpeed-VisualChat shows superior
scalability up to 70B parameter language model size, representing a significant
advancement in multi-modal language models and setting a solid foundation for
future explorations.
VidChapters-7M: Video Chapters at Scale
Authors: name: Antoine Yang, name: Arsha Nagrani, name: Ivan Laptev, name: Josef Sivic, name: Cordelia Schmid
Published: 2023-09-25 08:38:11

Segmenting long videos into chapters enables users to quickly navigate to the
information of their interest. This important topic has been understudied due
to the lack of publicly released datasets. To address this issue, we present
VidChapters-7M, a dataset of 817K user-chaptered videos including 7M chapters
in total. VidChapters-7M is automatically created from videos online in a
scalable manner by scraping user-annotated chapters and hence without any
additional manual annotation. We introduce the following three tasks based on
this data. First, the video chapter generation task consists of temporally
segmenting the video and generating a chapter title for each segment. To
further dissect the problem, we also define two variants of this task: video
chapter generation given ground-truth boundaries, which requires generating a
chapter title given an annotated video segment, and video chapter grounding,
which requires temporally localizing a chapter given its annotated title. We
benchmark both simple baselines and state-of-the-art video-language models for
these three tasks. We also show that pretraining on VidChapters-7M transfers
well to dense video captioning tasks in both zero-shot and finetuning settings,
largely improving the state of the art on the YouCook2 and ViTT benchmarks.
Finally, our experiments reveal that downstream performance scales well with
the size of the pretraining dataset. Our dataset, code, and models are publicly
available at https://antoyang.github.io/vidchapters.html.
Exploring Large Language Models' Cognitive Moral Development through
  Defining Issues Test
Authors: name: Kumar Tanmay, name: Aditi Khandelwal, name: Utkarsh Agarwal, name: Monojit Choudhury
Published: 2023-09-23 12:17:10

The development of large language models has instilled widespread interest
among the researchers to understand their inherent reasoning and
problem-solving capabilities. Despite good amount of research going on to
elucidate these capabilities, there is a still an appreciable gap in
understanding moral development and judgments of these models. The current
approaches of evaluating the ethical reasoning abilities of these models as a
classification task pose numerous inaccuracies because of over-simplification.
In this study, we built a psychological connection by bridging two disparate
fields-human psychology and AI. We proposed an effective evaluation framework
which can help to delineate the model's ethical reasoning ability in terms of
moral consistency and Kohlberg's moral development stages with the help of
Psychometric Assessment Tool-Defining Issues Test.
Calibrating LLM-Based Evaluator
Authors: name: Yuxuan Liu, name: Tianchi Yang, name: Shaohan Huang, name: Zihan Zhang, name: Haizhen Huang, name: Furu Wei, name: Weiwei Deng, name: Feng Sun, name: Qi Zhang
Published: 2023-09-23 08:46:11

Recent advancements in large language models (LLMs) on language modeling and
emergent capabilities make them a promising reference-free evaluator of natural
language generation quality, and a competent alternative to human evaluation.
However, hindered by the closed-source or high computational demand to host and
tune, there is a lack of practice to further calibrate an off-the-shelf
LLM-based evaluator towards better human alignment. In this work, we propose
AutoCalibrate, a multi-stage, gradient-free approach to automatically calibrate
and align an LLM-based evaluator toward human preference. Instead of explicitly
modeling human preferences, we first implicitly encompass them within a set of
human labels. Then, an initial set of scoring criteria is drafted by the
language model itself, leveraging in-context learning on different few-shot
examples. To further calibrate this set of criteria, we select the best
performers and re-draft them with self-refinement. Our experiments on multiple
text quality evaluation datasets illustrate a significant improvement in
correlation with expert evaluation through calibration. Our comprehensive
qualitative analysis conveys insightful intuitions and observations on the
essence of effective scoring criteria.
SCREWS: A Modular Framework for Reasoning with Revisions
Authors: name: Kumar Shridhar, name: Harsh Jhamtani, name: Hao Fang, name: Benjamin Van Durme, name: Jason Eisner, name: Patrick Xia
Published: 2023-09-20 15:59:54

Large language models (LLMs) can improve their accuracy on various tasks
through iteratively refining and revising their output based on feedback. We
observe that these revisions can introduce errors, in which case it is better
to roll back to a previous result. Further, revisions are typically
homogeneous: they use the same reasoning method that produced the initial
answer, which may not correct errors. To enable exploration in this space, we
present SCREWS, a modular framework for reasoning with revisions. It is
comprised of three main modules: Sampling, Conditional Resampling, and
Selection, each consisting of sub-modules that can be hand-selected per task.
We show that SCREWS not only unifies several previous approaches under a common
framework, but also reveals several novel strategies for identifying improved
reasoning chains. We evaluate our framework with state-of-the-art LLMs (ChatGPT
and GPT-4) on a diverse set of reasoning tasks and uncover useful new reasoning
strategies for each: arithmetic word problems, multi-hop question answering,
and code debugging. Heterogeneous revision strategies prove to be important, as
does selection between original and revised candidates.
Robotic Offline RL from Internet Videos via Value-Function Pre-Training
Authors: name: Chethan Bhateja, name: Derek Guo, name: Dibya Ghosh, name: Anikait Singh, name: Manan Tomar, name: Quan Vuong, name: Yevgen Chebotar, name: Sergey Levine, name: Aviral Kumar
Published: 2023-09-22 17:59:14

Pre-training on Internet data has proven to be a key ingredient for broad
generalization in many modern ML systems. What would it take to enable such
capabilities in robotic reinforcement learning (RL)? Offline RL methods, which
learn from datasets of robot experience, offer one way to leverage prior data
into the robotic learning pipeline. However, these methods have a "type
mismatch" with video data (such as Ego4D), the largest prior datasets available
for robotics, since video offers observation-only experience without the action
or reward annotations needed for RL methods. In this paper, we develop a system
for leveraging large-scale human video datasets in robotic offline RL, based
entirely on learning value functions via temporal-difference learning. We show
that value learning on video datasets learns representations that are more
conducive to downstream robotic offline RL than other approaches for learning
from video data. Our system, called V-PTR, combines the benefits of
pre-training on video data with robotic offline RL approaches that train on
diverse robot data, resulting in value functions and policies for manipulation
tasks that perform better, act robustly, and generalize broadly. On several
manipulation tasks on a real WidowX robot, our framework produces policies that
greatly improve over prior methods. Our video and additional details can be
found at https://dibyaghosh.com/vptr/