Orionnnn/hub-stats huggingface dataset:
Rethinking Compression: Reduced Order Modelling of Latent Features in
  Large Language Models
Authors: name: Arnav Chavan, name: Nahush Lele, name: Deepak Gupta
Published: 2023-12-12 07:56:57

Due to the substantial scale of Large Language Models (LLMs), the direct
application of conventional compression methodologies proves impractical. The
computational demands associated with even minimal gradient updates present
challenges, particularly on consumer-grade hardware. This paper introduces an
innovative approach for the parametric and practical compression of LLMs based
on reduced order modelling, which entails low-rank decomposition within the
feature space and re-parameterization in the weight space. Notably, this
compression technique operates in a layer-wise manner, obviating the need for a
GPU device and enabling the compression of billion-scale models within
stringent constraints of both memory and time. Our method represents a
significant advancement in model compression by leveraging matrix
decomposition, demonstrating superior efficacy compared to the prevailing
state-of-the-art structured pruning method.
Alignment for Honesty
Authors: name: Yuqing Yang, name: Ethan Chern, name: Xipeng Qiu, name: Graham Neubig, name: Pengfei Liu
Published: 2023-12-12 06:10:42

Recent research has made significant strides in applying alignment techniques
to enhance the helpfulness and harmlessness of large language models (LLMs) in
accordance with human intentions. In this paper, we argue for the importance of
alignment for honesty, ensuring that LLMs proactively refuse to answer
questions when they lack knowledge, while still not being overly conservative.
However, a pivotal aspect of alignment for honesty involves discerning the
limits of an LLM's knowledge, which is far from straightforward. This challenge
demands comprehensive solutions in terms of metric development, benchmark
creation, and training methodologies. In this paper, we address these
challenges by first establishing a precise problem definition and defining
``honesty'' inspired by the Analects of Confucius. This serves as a cornerstone
for developing metrics that effectively measure an LLM's honesty by quantifying
its progress post-alignment. Furthermore, we introduce a flexible training
framework which is further instantiated by several efficient fine-tuning
techniques that emphasize honesty without sacrificing performance on other
tasks. Our extensive experiments reveal that these aligned models show a marked
increase in honesty, as indicated by our proposed metrics. We open-source a
wealth of resources to facilitate future research at
https://github.com/GAIR-NLP/alignment-for-honesty, including honesty-aligned
models, training and evaluation datasets for honesty alignment, concept
glossary, as well as all relevant source code.
"I Want It That Way": Enabling Interactive Decision Support Using Large
  Language Models and Constraint Programming
Authors: name: Connor Lawless, name: Jakob Schoeffer, name: Lindy Le, name: Kael Rowan, name: Shilad Sen, name: Cristina St. Hill, name: Jina Suh, name: Bahar Sarrafzadeh
Published: 2023-12-12 00:39:24

A critical factor in the success of decision support systems is the accurate
modeling of user preferences. Psychology research has demonstrated that users
often develop their preferences during the elicitation process, highlighting
the pivotal role of system-user interaction in developing personalized systems.
This paper introduces a novel approach, combining Large Language Models (LLMs)
with Constraint Programming to facilitate interactive decision support. We
study this hybrid framework through the lens of meeting scheduling, a
time-consuming daily activity faced by a multitude of information workers. We
conduct three studies to evaluate the novel framework, including a diary study
(n=64) to characterize contextual scheduling preferences, a quantitative
evaluation of the system's performance, and a user study (n=10) with a
prototype system. Our work highlights the potential for a hybrid LLM and
optimization approach for iterative preference elicitation and design
considerations for building systems that support human-system collaborative
decision-making processes.
Honeybee: Locality-enhanced Projector for Multimodal LLM
Authors: name: Junbum Cha, name: Wooyoung Kang, name: Jonghwan Mun, name: Byungseok Roh
Published: 2023-12-11 18:59:06

In Multimodal Large Language Models (MLLMs), a visual projector plays a
crucial role in bridging pre-trained vision encoders with LLMs, enabling
profound visual understanding while harnessing the LLMs' robust capabilities.
Despite the importance of the visual projector, it has been relatively less
explored. In this study, we first identify two essential projector properties:
(i) flexibility in managing the number of visual tokens, crucial for MLLMs'
overall efficiency, and (ii) preservation of local context from visual
features, vital for spatial understanding. Based on these findings, we propose
a novel projector design that is both flexible and locality-enhanced,
effectively satisfying the two desirable properties. Additionally, we present
comprehensive strategies to effectively utilize multiple and multifaceted
instruction datasets. Through extensive experiments, we examine the impact of
individual design choices. Finally, our proposed MLLM, Honeybee, remarkably
outperforms previous state-of-the-art methods across various benchmarks,
including MME, MMBench, SEED-Bench, and LLaVA-Bench, achieving significantly
higher efficiency. Code and models are available at
https://github.com/kakaobrain/honeybee.
Steering Llama 2 via Contrastive Activation Addition
Authors: name: Nina Rimsky, name: Nick Gabrieli, name: Julian Schulz, name: Meg Tong, name: Evan Hubinger, name: Alexander Matt Turner
Published: 2023-12-09 04:40:46

We introduce Contrastive Activation Addition (CAA), an innovative method for
steering language models by modifying activations during their forward passes.
CAA computes ``steering vectors'' by averaging the difference in residual
stream activations between pairs of positive and negative examples of a
particular behavior such as factual versus hallucinatory responses. During
inference, these steering vectors are added at all token positions after the
user's prompt with either a positive or negative coefficient, allowing precise
control over the degree of the targeted behavior. We evaluate CAA's
effectiveness on Llama 2 Chat using both multiple-choice behavioral question
datasets and open-ended generation tasks. We demonstrate that CAA significantly
alters model behavior, outperforms traditional methods like finetuning and
few-shot prompting, and minimally reduces capabilities. Moreover, by employing
various activation space interpretation methods, we gain deeper insights into
CAA's mechanisms. CAA both accurately steers model outputs and also sheds light
on how high-level concepts are represented in Large Language Models (LLMs).
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
Authors: name: Hakan Inan, name: Kartikeya Upasani, name: Jianfeng Chi, name: Rashi Rungta, name: Krithika Iyer, name: Yuning Mao, name: Michael Tontchev, name: Qing Hu, name: Brian Fuller, name: Davide Testuggine, name: Madian Khabsa
Published: 2023-12-07 19:40:50

We introduce Llama Guard, an LLM-based input-output safeguard model geared
towards Human-AI conversation use cases. Our model incorporates a safety risk
taxonomy, a valuable tool for categorizing a specific set of safety risks found
in LLM prompts (i.e., prompt classification). This taxonomy is also
instrumental in classifying the responses generated by LLMs to these prompts, a
process we refer to as response classification. For the purpose of both prompt
and response classification, we have meticulously gathered a dataset of high
quality. Llama Guard, a Llama2-7b model that is instruction-tuned on our
collected dataset, albeit low in volume, demonstrates strong performance on
existing benchmarks such as the OpenAI Moderation Evaluation dataset and
ToxicChat, where its performance matches or exceeds that of currently available
content moderation tools. Llama Guard functions as a language model, carrying
out multi-class classification and generating binary decision scores.
Furthermore, the instruction fine-tuning of Llama Guard allows for the
customization of tasks and the adaptation of output formats. This feature
enhances the model's capabilities, such as enabling the adjustment of taxonomy
categories to align with specific use cases, and facilitating zero-shot or
few-shot prompting with diverse taxonomies at the input. We are making Llama
Guard model weights available and we encourage researchers to further develop
and adapt them to meet the evolving needs of the community for AI safety.
Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D
  Prior
Authors: name: Fangfu Liu, name: Diankun Wu, name: Yi Wei, name: Yongming Rao, name: Yueqi Duan
Published: 2023-12-11 18:59:18

Recently, 3D content creation from text prompts has demonstrated remarkable
progress by utilizing 2D and 3D diffusion models. While 3D diffusion models
ensure great multi-view consistency, their ability to generate high-quality and
diverse 3D assets is hindered by the limited 3D data. In contrast, 2D diffusion
models find a distillation approach that achieves excellent generalization and
rich details without any 3D data. However, 2D lifting methods suffer from
inherent view-agnostic ambiguity thereby leading to serious multi-face Janus
issues, where text prompts fail to provide sufficient guidance to learn
coherent 3D results. Instead of retraining a costly viewpoint-aware model, we
study how to fully exploit easily accessible coarse 3D knowledge to enhance the
prompts and guide 2D lifting optimization for refinement. In this paper, we
propose Sherpa3D, a new text-to-3D framework that achieves high-fidelity,
generalizability, and geometric consistency simultaneously. Specifically, we
design a pair of guiding strategies derived from the coarse 3D prior generated
by the 3D diffusion model: a structural guidance for geometric fidelity and a
semantic guidance for 3D coherence. Employing the two types of guidance, the 2D
diffusion model enriches the 3D content with diversified and high-quality
results. Extensive experiments show the superiority of our Sherpa3D over the
state-of-the-art text-to-3D methods in terms of quality and 3D consistency.
Photorealistic Video Generation with Diffusion Models
Authors: name: Agrim Gupta, name: Lijun Yu, name: Kihyuk Sohn, name: Xiuye Gu, name: Meera Hahn, name: Li Fei-Fei, name: Irfan Essa, name: Lu Jiang, name: José Lezama
Published: 2023-12-11 18:59:57

We present W.A.L.T, a transformer-based approach for photorealistic video
generation via diffusion modeling. Our approach has two key design decisions.
First, we use a causal encoder to jointly compress images and videos within a
unified latent space, enabling training and generation across modalities.
Second, for memory and training efficiency, we use a window attention
architecture tailored for joint spatial and spatiotemporal generative modeling.
Taken together these design decisions enable us to achieve state-of-the-art
performance on established video (UCF-101 and Kinetics-600) and image
(ImageNet) generation benchmarks without using classifier free guidance.
Finally, we also train a cascade of three models for the task of text-to-video
generation consisting of a base latent video diffusion model, and two video
super-resolution diffusion models to generate videos of 512 times 896
resolution at 8 frames per second.
Efficient Quantization Strategies for Latent Diffusion Models
Authors: name: Yuewei Yang, name: Xiaoliang Dai, name: Jialiang Wang, name: Peizhao Zhang, name: Hongbo Zhang
Published: 2023-12-09 01:47:16

Latent Diffusion Models (LDMs) capture the dynamic evolution of latent
variables over time, blending patterns and multimodality in a generative
system. Despite the proficiency of LDM in various applications, such as
text-to-image generation, facilitated by robust text encoders and a variational
autoencoder, the critical need to deploy large generative models on edge
devices compels a search for more compact yet effective alternatives. Post
Training Quantization (PTQ), a method to compress the operational size of deep
learning models, encounters challenges when applied to LDM due to temporal and
structural complexities. This study proposes a quantization strategy that
efficiently quantize LDMs, leveraging Signal-to-Quantization-Noise Ratio (SQNR)
as a pivotal metric for evaluation. By treating the quantization discrepancy as
relative noise and identifying sensitive part(s) of a model, we propose an
efficient quantization approach encompassing both global and local strategies.
The global quantization process mitigates relative quantization noise by
initiating higher-precision quantization on sensitive blocks, while local
treatments address specific challenges in quantization-sensitive and
time-sensitive modules. The outcomes of our experiments reveal that the
implementation of both global and local treatments yields a highly efficient
and effective Post Training Quantization (PTQ) of LDMs.
Beyond Human Data: Scaling Self-Training for Problem-Solving with
  Language Models
Authors: name: Avi Singh, name: John D. Co-Reyes, name: Rishabh Agarwal, name: Ankesh Anand, name: Piyush Patil, name: Peter J. Liu, name: James Harrison, name: Jaehoon Lee, name: Kelvin Xu, name: Aaron Parisi, name: Abhishek Kumar, name: Alex Alemi, name: Alex Rizkowsky, name: Azade Nova, name: Ben Adlam, name: Bernd Bohnet, name: Hanie Sedghi, name: Igor Mordatch, name: Isabelle Simpson, name: Izzeddin Gur, name: Jasper Snoek, name: Jeffrey Pennington, name: Jiri Hron, name: Kathleen Kenealy, name: Kevin Swersky, name: Kshiteej Mahajan, name: Laura Culp, name: Lechao Xiao, name: Maxwell L. Bileschi, name: Noah Constant, name: Roman Novak, name: Rosanne Liu, name: Tris Warkentin, name: Yundi Qian, name: Ethan Dyer, name: Behnam Neyshabur, name: Jascha Sohl-Dickstein, name: Noah Fiedel
Published: 2023-12-11 18:17:43

Fine-tuning language models~(LMs) on human-generated data remains a prevalent
practice. However, the performance of such models is often limited by the
quantity and diversity of high-quality human data. In this paper, we explore
whether we can go beyond human data on tasks where we have access to scalar
feedback, for example, on math problems where one can verify correctness. To do
so, we investigate a simple self-training method based on
expectation-maximization, which we call ReST^{EM}, where we (1) generate
samples from the model and filter them using binary feedback, (2) fine-tune the
model on these samples, and (3) repeat this process a few times. Testing on
advanced MATH reasoning and APPS coding benchmarks using PaLM-2 models, we find
that ReST^{EM} scales favorably with model size and significantly surpasses
fine-tuning only on human data. Overall, our findings suggest self-training
with feedback can substantially reduce dependence on human-generated data.
LLM360: Towards Fully Transparent Open-Source LLMs
Authors: name: Zhengzhong Liu, name: Aurick Qiao, name: Willie Neiswanger, name: Hongyi Wang, name: Bowen Tan, name: Tianhua Tao, name: Junbo Li, name: Yuqi Wang, name: Suqi Sun, name: Omkar Pangarkar, name: Richard Fan, name: Yi Gu, name: Victor Miller, name: Yonghao Zhuang, name: Guowei He, name: Haonan Li, name: Fajri Koto, name: Liping Tang, name: Nikhil Ranjan, name: Zhiqiang Shen, name: Xuguang Ren, name: Roberto Iriondo, name: Cun Mu, name: Zhiting Hu, name: Mark Schulze, name: Preslav Nakov, name: Tim Baldwin, name: Eric P. Xing
Published: 2023-12-11 17:39:00

The recent surge in open-source Large Language Models (LLMs), such as LLaMA,
Falcon, and Mistral, provides diverse options for AI practitioners and
researchers. However, most LLMs have only released partial artifacts, such as
the final model weights or inference code, and technical reports increasingly
limit their scope to high-level design choices and surface statistics. These
choices hinder progress in the field by degrading transparency into the
training of LLMs and forcing teams to rediscover many details in the training
process. We present LLM360, an initiative to fully open-source LLMs, which
advocates for all training code and data, model checkpoints, and intermediate
results to be made available to the community. The goal of LLM360 is to support
open and collaborative AI research by making the end-to-end LLM training
process transparent and reproducible by everyone. As a first step of LLM360, we
release two 7B parameter LLMs pre-trained from scratch, Amber and CrystalCoder,
including their training code, data, intermediate checkpoints, and analyses (at
https://www.llm360.ai). We are committed to continually pushing the boundaries
of LLMs through this open-source effort. More large-scale and stronger models
are underway and will be released in the future.
Federated Full-Parameter Tuning of Billion-Sized Language Models with
  Communication Cost under 18 Kilobytes
Authors: name: Zhen Qin, name: Daoyuan Chen, name: Bingchen Qian, name: Bolin Ding, name: Yaliang Li, name: Shuiguang Deng
Published: 2023-12-11 13:03:21

Pre-trained large language models (LLMs) require fine-tuning to improve their
responsiveness to natural language instructions. Federated learning (FL) offers
a way to perform fine-tuning using the abundant data on end devices without
compromising data privacy. Most existing federated fine-tuning methods for LLMs
rely on parameter-efficient fine-tuning techniques, which may not reach the
performance heights possible with full-parameter tuning. However, the
communication overhead associated with full-parameter tuning is prohibitively
high for both servers and clients. This work introduces FedKSeed, a novel
approach that employs zeroth-order optimization (ZOO) with a set of random
seeds. It enables federated full-parameter tuning of billion-sized LLMs
directly on devices. Our method significantly reduces transmission requirements
between the server and clients to just a few scalar gradients and random seeds,
amounting to only a few thousand bytes. Building on this, we develop a strategy
to assess the significance of ZOO perturbations for FL, allowing for
probability-differentiated seed sampling. This prioritizes perturbations that
have a greater impact on model accuracy. Experiments across six scenarios with
different LLMs, datasets and data partitions demonstrate that our approach
outperforms existing federated LLM fine-tuning methods in terms of both
communication efficiency and new task generalization.
Evaluation of Large Language Models for Decision Making in Autonomous
  Driving
Authors: name: Kotaro Tanahashi, name: Yuichi Inoue, name: Yu Yamaguchi, name: Hidetatsu Yaginuma, name: Daiki Shiotsuka, name: Hiroyuki Shimatani, name: Kohei Iwamasa, name: Yoshiaki Inoue, name: Takafumi Yamaguchi, name: Koki Igari, name: Tsukasa Horinouchi, name: Kento Tokuhiro, name: Yugo Tokuchi, name: Shunsuke Aoki
Published: 2023-12-11 12:56:40

Various methods have been proposed for utilizing Large Language Models (LLMs)
in autonomous driving. One strategy of using LLMs for autonomous driving
involves inputting surrounding objects as text prompts to the LLMs, along with
their coordinate and velocity information, and then outputting the subsequent
movements of the vehicle. When using LLMs for such purposes, capabilities such
as spatial recognition and planning are essential. In particular, two
foundational capabilities are required: (1) spatial-aware decision making,
which is the ability to recognize space from coordinate information and make
decisions to avoid collisions, and (2) the ability to adhere to traffic rules.
However, quantitative research has not been conducted on how accurately
different types of LLMs can handle these problems. In this study, we
quantitatively evaluated these two abilities of LLMs in the context of
autonomous driving. Furthermore, to conduct a Proof of Concept (POC) for the
feasibility of implementing these abilities in actual vehicles, we developed a
system that uses LLMs to drive a vehicle.
Order Matters in the Presence of Dataset Imbalance for Multilingual
  Learning
Authors: name: Dami Choi, name: Derrick Xin, name: Hamid Dadkhahi, name: Justin Gilmer, name: Ankush Garg, name: Orhan Firat, name: Chih-Kuan Yeh, name: Andrew M. Dai, name: Behrooz Ghorbani
Published: 2023-12-11 05:46:57

In this paper, we empirically study the optimization dynamics of multi-task
learning, particularly focusing on those that govern a collection of tasks with
significant data imbalance. We present a simple yet effective method of
pre-training on high-resource tasks, followed by fine-tuning on a mixture of
high/low-resource tasks. We provide a thorough empirical study and analysis of
this method's benefits showing that it achieves consistent improvements
relative to the performance trade-off profile of standard static weighting. We
analyze under what data regimes this method is applicable and show its
improvements empirically in neural machine translation (NMT) and multi-lingual
language modeling.
Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models
Authors: name: Haoran Wei, name: Lingyu Kong, name: Jinyue Chen, name: Liang Zhao, name: Zheng Ge, name: Jinrong Yang, name: Jianjian Sun, name: Chunrui Han, name: Xiangyu Zhang
Published: 2023-12-11 04:26:17

Modern Large Vision-Language Models (LVLMs) enjoy the same vision vocabulary
-- CLIP, which can cover most common vision tasks. However, for some special
vision task that needs dense and fine-grained vision perception, e.g.,
document-level OCR or chart understanding, especially in non-English scenarios,
the CLIP-style vocabulary may encounter low efficiency in tokenizing the vision
knowledge and even suffer out-of-vocabulary problem. Accordingly, we propose
Vary, an efficient and effective method to scale up the vision vocabulary of
LVLMs. The procedures of Vary are naturally divided into two folds: the
generation and integration of a new vision vocabulary. In the first phase, we
devise a vocabulary network along with a tiny decoder-only transformer to
produce the desired vocabulary via autoregression. In the next, we scale up the
vanilla vision vocabulary by merging the new one with the original one (CLIP),
enabling the LVLMs can quickly garner new features. Compared to the popular
BLIP-2, MiniGPT4, and LLaVA, Vary can maintain its vanilla capabilities while
enjoying more excellent fine-grained perception and understanding ability.
Specifically, Vary is competent in new document parsing features (OCR or
markdown conversion) while achieving 78.2% ANLS in DocVQA and 36.2% in MMVet.
Our code will be publicly available on the homepage.