Orionnnn/hub-stats huggingface dataset:
Specific versus General Principles for Constitutional AI
Authors: name: Sandipan Kundu, name: Yuntao Bai, name: Saurav Kadavath, name: Amanda Askell, name: Andrew Callahan, name: Anna Chen, name: Anna Goldie, name: Avital Balwit, name: Azalia Mirhoseini, name: Brayden McLean, name: Catherine Olsson, name: Cassie Evraets, name: Eli Tran-Johnson, name: Esin Durmus, name: Ethan Perez, name: Jackson Kernion, name: Jamie Kerr, name: Kamal Ndousse, name: Karina Nguyen, name: Nelson Elhage, name: Newton Cheng, name: Nicholas Schiefer, name: Nova DasSarma, name: Oliver Rausch, name: Robin Larson, name: Shannon Yang, name: Shauna Kravec, name: Timothy Telleen-Lawton, name: Thomas I. Liao, name: Tom Henighan, name: Tristan Hume, name: Zac Hatfield-Dodds, name: Sören Mindermann, name: Nicholas Joseph, name: Sam McCandlish, name: Jared Kaplan
Published: 2023-10-20 20:12:45

Human feedback can prevent overtly harmful utterances in conversational
models, but may not automatically mitigate subtle problematic behaviors such as
a stated desire for self-preservation or power. Constitutional AI offers an
alternative, replacing human feedback with feedback from AI models conditioned
only on a list of written principles. We find this approach effectively
prevents the expression of such behaviors. The success of simple principles
motivates us to ask: can models learn general ethical behaviors from only a
single written principle? To test this, we run experiments using a principle
roughly stated as "do what's best for humanity". We find that the largest
dialogue models can generalize from this short constitution, resulting in
harmless assistants with no stated interest in specific motivations like power.
A general principle may thus partially avoid the need for a long list of
constitutions targeting potentially harmful behaviors. However, more detailed
constitutions still improve fine-grained control over specific types of harms.
This suggests both general and specific principles have value for steering AI
safely.
Localizing and Editing Knowledge in Text-to-Image Generative Models
Authors: name: Samyadeep Basu, name: Nanxuan Zhao, name: Vlad Morariu, name: Soheil Feizi, name: Varun Manjunatha
Published: 2023-10-20 17:31:12

Text-to-Image Diffusion Models such as Stable-Diffusion and Imagen have
achieved unprecedented quality of photorealism with state-of-the-art FID scores
on MS-COCO and other generation benchmarks. Given a caption, image generation
requires fine-grained knowledge about attributes such as object structure,
style, and viewpoint amongst others. Where does this information reside in
text-to-image generative models? In our paper, we tackle this question and
understand how knowledge corresponding to distinct visual attributes is stored
in large-scale text-to-image diffusion models. We adapt Causal Mediation
Analysis for text-to-image models and trace knowledge about distinct visual
attributes to various (causal) components in the (i) UNet and (ii) text-encoder
of the diffusion model. In particular, we show that unlike generative
large-language models, knowledge about different attributes is not localized in
isolated components, but is instead distributed amongst a set of components in
the conditional UNet. These sets of components are often distinct for different
visual attributes. Remarkably, we find that the CLIP text-encoder in public
text-to-image models such as Stable-Diffusion contains only one causal state
across different visual attributes, and this is the first self-attention layer
corresponding to the last subject token of the attribute in the caption. This
is in stark contrast to the causal states in other language models which are
often the mid-MLP layers. Based on this observation of only one causal state in
the text-encoder, we introduce a fast, data-free model editing method
Diff-QuickFix which can effectively edit concepts in text-to-image models.
DiffQuickFix can edit (ablate) concepts in under a second with a closed-form
update, providing a significant 1000x speedup and comparable editing
performance to existing fine-tuning based editing methods.
DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual
  Design
Authors: name: Kevin Lin, name: Zhengyuan Yang, name: Linjie Li, name: Jianfeng Wang, name: Lijuan Wang
Published: 2023-10-23 17:48:38

We introduce DEsignBench, a text-to-image (T2I) generation benchmark tailored
for visual design scenarios. Recent T2I models like DALL-E 3 and others, have
demonstrated remarkable capabilities in generating photorealistic images that
align closely with textual inputs. While the allure of creating visually
captivating images is undeniable, our emphasis extends beyond mere aesthetic
pleasure. We aim to investigate the potential of using these powerful models in
authentic design contexts. In pursuit of this goal, we develop DEsignBench,
which incorporates test samples designed to assess T2I models on both "design
technical capability" and "design application scenario." Each of these two
dimensions is supported by a diverse set of specific design categories. We
explore DALL-E 3 together with other leading T2I models on DEsignBench,
resulting in a comprehensive visual gallery for side-by-side comparisons. For
DEsignBench benchmarking, we perform human evaluations on generated images in
DEsignBench gallery, against the criteria of image-text alignment, visual
aesthetic, and design creativity. Our evaluation also considers other
specialized design capabilities, including text rendering, layout composition,
color harmony, 3D design, and medium style. In addition to human evaluations,
we introduce the first automatic image generation evaluator powered by GPT-4V.
This evaluator provides ratings that align well with human judgments, while
being easily replicable and cost-efficient. A high-resolution version is
available at
https://github.com/design-bench/design-bench.github.io/raw/main/designbench.pdf?download=
Branch-Solve-Merge Improves Large Language Model Evaluation and
  Generation
Authors: name: Swarnadeep Saha, name: Omer Levy, name: Asli Celikyilmaz, name: Mohit Bansal, name: Jason Weston, name: Xian Li
Published: 2023-10-23 17:29:48

Large Language Models (LLMs) are frequently used for multi-faceted language
generation and evaluation tasks that involve satisfying intricate user
constraints or taking into account multiple aspects and criteria. However,
their performance can fall short, due to the model's lack of coherence and
inability to plan and decompose the problem. We propose Branch-Solve-Merge
(BSM), a Large Language Model program (Schlag et al., 2023) for tackling such
challenging natural language tasks. It consists of branch, solve, and merge
modules that are parameterized with specific prompts to the base LLM. These
three modules plan a decomposition of the task into multiple parallel
sub-tasks, independently solve them, and fuse the solutions to the sub-tasks.
We apply our method to the tasks of LLM response evaluation and constrained
text generation and evaluate its effectiveness with multiple LLMs, including
Vicuna, LLaMA-2-chat, and GPT-4. BSM improves the evaluation correctness and
consistency for each LLM by enhancing human-LLM agreement by up to 26%,
reducing length and pairwise position biases by up to 50%, and allowing
LLaMA-2-chat to match or outperform GPT-4 on most domains. On the constraint
story generation task, BSM improves the coherence of the stories while also
improving constraint satisfaction by 12%.
Exploring the Boundaries of GPT-4 in Radiology
Authors: name: Qianchu Liu, name: Stephanie Hyland, name: Shruthi Bannur, name: Kenza Bouzid, name: Daniel C. Castro, name: Maria Teodora Wetscherek, name: Robert Tinn, name: Harshita Sharma, name: Fernando Pérez-García, name: Anton Schwaighofer, name: Pranav Rajpurkar, name: Sameer Tajdin Khanna, name: Hoifung Poon, name: Naoto Usuyama, name: Anja Thieme, name: Aditya V. Nori, name: Matthew P. Lungren, name: Ozan Oktay, name: Javier Alvarez-Valle
Published: 2023-10-23 05:13:03

The recent success of general-domain large language models (LLMs) has
significantly changed the natural language processing paradigm towards a
unified foundation model across domains and applications. In this paper, we
focus on assessing the performance of GPT-4, the most capable LLM so far, on
the text-based applications for radiology reports, comparing against
state-of-the-art (SOTA) radiology-specific models. Exploring various prompting
strategies, we evaluated GPT-4 on a diverse range of common radiology tasks and
we found GPT-4 either outperforms or is on par with current SOTA radiology
models. With zero-shot prompting, GPT-4 already obtains substantial gains
(approx 10% absolute improvement) over radiology models in temporal sentence
similarity classification (accuracy) and natural language inference (F_1).
For tasks that require learning dataset-specific style or schema (e.g. findings
summarisation), GPT-4 improves with example-based prompting and matches
supervised SOTA. Our extensive error analysis with a board-certified
radiologist shows GPT-4 has a sufficient level of radiology knowledge with only
occasional errors in complex context that require nuanced domain knowledge. For
findings summarisation, GPT-4 outputs are found to be overall comparable with
existing manually-written impressions.
HallusionBench: You See What You Think? Or You Think What You See? An
  Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5,
  and Other Multi-modality Models
Authors: name: Fuxiao Liu, name: Tianrui Guan, name: Zongxia Li, name: Lichang Chen, name: Yaser Yacoob, name: Dinesh Manocha, name: Tianyi Zhou
Published: 2023-10-23 04:49:09

Large language models (LLMs), after being aligned with vision models and
integrated into vision-language models (VLMs), can bring impressive improvement
in image reasoning tasks. This was shown by the recently released GPT-4V(ison),
LLaVA-1.5, etc. However, the strong language prior in these SOTA LVLMs can be a
double-edged sword: they may ignore the image context and solely rely on the
(even contradictory) language prior for reasoning. In contrast, the vision
modules in VLMs are weaker than LLMs and may result in misleading visual
representations, which are then translated to confident mistakes by LLMs. To
study these two types of VLM mistakes, i.e., language hallucination and visual
illusion, we curated HallusionBench, an image-context reasoning benchmark that
is still challenging to even GPT-4V and LLaVA-1.5. We provide a detailed
analysis of examples in HallusionBench, which sheds novel insights on the
illusion or hallucination of VLMs and how to improve them in the future. The
benchmark and codebase will be released at
https://github.com/tianyi-lab/HallusionBench.
InstructExcel: A Benchmark for Natural Language Instruction in Excel
Authors: name: Justin Payan, name: Swaroop Mishra, name: Mukul Singh, name: Carina Negreanu, name: Christian Poelitz, name: Chitta Baral, name: Subhro Roy, name: Rasika Chakravarthy, name: Benjamin Van Durme, name: Elnaz Nouri
Published: 2023-10-23 02:00:55

With the evolution of Large Language Models (LLMs) we can solve increasingly
more complex NLP tasks across various domains, including spreadsheets. This
work investigates whether LLMs can generate code (Excel OfficeScripts, a
TypeScript API for executing many tasks in Excel) that solves Excel specific
tasks provided via natural language user instructions. To do so we introduce a
new large-scale benchmark, InstructExcel, created by leveraging the 'Automate'
feature in Excel to automatically generate OfficeScripts from users' actions.
Our benchmark includes over 10k samples covering 170+ Excel operations across
2,000 publicly available Excel spreadsheets. Experiments across various
zero-shot and few-shot settings show that InstructExcel is a hard benchmark for
state of the art models like GPT-4. We observe that (1) using GPT-4 over
GPT-3.5, (2) providing more in-context examples, and (3) dynamic prompting can
help improve performance on this benchmark.
DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model
  Statistics
Authors: name: Kaiwen Zheng, name: Cheng Lu, name: Jianfei Chen, name: Jun Zhu
Published: 2023-10-20 04:23:12

Diffusion probabilistic models (DPMs) have exhibited excellent performance
for high-fidelity image generation while suffering from inefficient sampling.
Recent works accelerate the sampling procedure by proposing fast ODE solvers
that leverage the specific ODE form of DPMs. However, they highly rely on
specific parameterization during inference (such as noise/data prediction),
which might not be the optimal choice. In this work, we propose a novel
formulation towards the optimal parameterization during sampling that minimizes
the first-order discretization error of the ODE solution. Based on such
formulation, we propose DPM-Solver-v3, a new fast ODE solver for DPMs
by introducing several coefficients efficiently computed on the pretrained
model, which we call empirical model statistics. We further
incorporate multistep methods and a predictor-corrector framework, and propose
some techniques for improving sample quality at small numbers of function
evaluations (NFE) or large guidance scales. Experiments show that DPM-Solver-v3
achieves consistently better or comparable performance in both unconditional
and conditional sampling with both pixel-space and latent-space DPMs,
especially in 5sim10 NFEs. We achieve FIDs of 12.21 (5 NFE), 2.51 (10 NFE)
on unconditional CIFAR10, and MSE of 0.55 (5 NFE, 7.5 guidance scale) on Stable
Diffusion, bringing a speed-up of 15\%sim30\% compared to previous
state-of-the-art training-free methods. Code is available at
https://github.com/thu-ml/DPM-Solver-v3.
ScaleLong: Towards More Stable Training of Diffusion Model via Scaling
  Network Long Skip Connection
Authors: name: Zhongzhan Huang, name: Pan Zhou, name: Shuicheng Yan, name: Liang Lin
Published: 2023-10-20 14:45:52

In diffusion models, UNet is the most popular network backbone, since its
long skip connects (LSCs) to connect distant network blocks can aggregate
long-distant information and alleviate vanishing gradient. Unfortunately, UNet
often suffers from unstable training in diffusion models which can be
alleviated by scaling its LSC coefficients smaller. However, theoretical
understandings of the instability of UNet in diffusion models and also the
performance improvement of LSC scaling remain absent yet. To solve this issue,
we theoretically show that the coefficients of LSCs in UNet have big effects on
the stableness of the forward and backward propagation and robustness of UNet.
Specifically, the hidden feature and gradient of UNet at any layer can
oscillate and their oscillation ranges are actually large which explains the
instability of UNet training. Moreover, UNet is also provably sensitive to
perturbed input, and predicts an output distant from the desired output,
yielding oscillatory loss and thus oscillatory gradient. Besides, we also
observe the theoretical benefits of the LSC coefficient scaling of UNet in the
stableness of hidden features and gradient and also robustness. Finally,
inspired by our theory, we propose an effective coefficient scaling framework
ScaleLong that scales the coefficients of LSC in UNet and better improves the
training stability of UNet. Experimental results on four famous datasets show
that our methods are superior to stabilize training and yield about 1.5x
training acceleration on different diffusion models with UNet or UViT
backbones. Code: https://github.com/sail-sg/ScaleLong
DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture
  Propagation
Authors: name: Bangbang Yang, name: Wenqi Dong, name: Lin Ma, name: Wenbo Hu, name: Xiao Liu, name: Zhaopeng Cui, name: Yuewen Ma
Published: 2023-10-19 19:29:23

Diffusion-based methods have achieved prominent success in generating 2D
media. However, accomplishing similar proficiencies for scene-level mesh
texturing in 3D spatial applications, e.g., XR/VR, remains constrained,
primarily due to the intricate nature of 3D geometry and the necessity for
immersive free-viewpoint rendering. In this paper, we propose a novel indoor
scene texturing framework, which delivers text-driven texture generation with
enchanting details and authentic spatial coherence. The key insight is to first
imagine a stylized 360{\deg} panoramic texture from the central viewpoint of
the scene, and then propagate it to the rest areas with inpainting and
imitating techniques. To ensure meaningful and aligned textures to the scene,
we develop a novel coarse-to-fine panoramic texture generation approach with
dual texture alignment, which both considers the geometry and texture cues of
the captured scenes. To survive from cluttered geometries during texture
propagation, we design a separated strategy, which conducts texture inpainting
in confidential regions and then learns an implicit imitating network to
synthesize textures in occluded and tiny structural areas. Extensive
experiments and the immersive VR application on real-world indoor scenes
demonstrate the high quality of the generated textures and the engaging
experience on VR headsets. Project webpage:
https://ybbbbt.com/publication/dreamspace
SILC: Improving Vision Language Pretraining with Self-Distillation
Authors: name: Muhammad Ferjad Naeem, name: Yongqin Xian, name: Xiaohua Zhai, name: Lukas Hoyer, name: Luc Van Gool, name: Federico Tombari
Published: 2023-10-20 08:44:47

Image-Text pretraining on web-scale image caption dataset has become the
default recipe for open vocabulary classification and retrieval models thanks
to the success of CLIP and its variants. Several works have also used CLIP
features for dense prediction tasks and have shown the emergence of open-set
abilities. However, the contrastive objective only focuses on image-text
alignment and does not incentivise image feature learning for dense prediction
tasks. In this work, we propose the simple addition of local-to-global
correspondence learning by self-distillation as an additional objective for
contrastive pre-training to propose SILC. We show that distilling local image
features from an exponential moving average (EMA) teacher model significantly
improves model performance on several computer vision tasks including
classification, retrieval, and especially segmentation. We further show that
SILC scales better with the same training duration compared to the baselines.
Our model SILC sets a new state of the art for zero-shot classification, few
shot classification, image and text retrieval, zero-shot segmentation, and open
vocabulary segmentation.
Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large
  Language Models by Extrapolating Errors from Small Models
Authors: name: Ruida Wang, name: Wangchunshu Zhou, name: Mrinmaya Sachan
Published: 2023-10-20 17:14:25

*Data Synthesis* is a promising way to train a small model with very little
labeled data. One approach for data synthesis is to leverage the rich knowledge
from large language models to synthesize pseudo training examples for small
models, making it possible to achieve both data and compute efficiency at the
same time. However, a key challenge in data synthesis is that the synthesized
dataset often suffers from a large distributional discrepancy from the *real
task* data distribution. Thus, in this paper, we propose *Synthesis Step by
Step* (**S3**), a data synthesis framework that shrinks this distribution gap
by iteratively extrapolating the errors made by a small model trained on the
synthesized dataset on a small real-world validation dataset using a large
language model. Extensive experiments on multiple NLP tasks show that our
approach improves the performance of a small model by reducing the gap between
the synthetic dataset and the real data, resulting in significant improvement
compared to several baselines: 9.48% improvement compared to ZeroGen and 2.73%
compared to GoldGen, and at most 15.17% improvement compared to the small model
trained on human-annotated data.
Contrastive Prefence Learning: Learning from Human Feedback without RL
Authors: name: Joey Hejna, name: Rafael Rafailov, name: Harshit Sikchi, name: Chelsea Finn, name: Scott Niekum, name: W. Bradley Knox, name: Dorsa Sadigh
Published: 2023-10-20 16:37:56

Reinforcement Learning from Human Feedback (RLHF) has emerged as a popular
paradigm for aligning models with human intent. Typically RLHF algorithms
operate in two phases: first, use human preferences to learn a reward function
and second, align the model by optimizing the learned reward via reinforcement
learning (RL). This paradigm assumes that human preferences are distributed
according to reward, but recent work suggests that they instead follow the
regret under the user's optimal policy. Thus, learning a reward function from
feedback is not only based on a flawed assumption of human preference, but also
leads to unwieldy optimization challenges that stem from policy gradients or
bootstrapping in the RL phase. Because of these optimization challenges,
contemporary RLHF methods restrict themselves to contextual bandit settings
(e.g., as in large language models) or limit observation dimensionality (e.g.,
state-based robotics). We overcome these limitations by introducing a new
family of algorithms for optimizing behavior from human feedback using the
regret-based model of human preferences. Using the principle of maximum
entropy, we derive Contrastive Preference Learning (CPL), an algorithm for
learning optimal policies from preferences without learning reward functions,
circumventing the need for RL. CPL is fully off-policy, uses only a simple
contrastive objective, and can be applied to arbitrary MDPs. This enables CPL
to elegantly scale to high-dimensional and sequential RLHF problems while being
simpler than prior methods.
Towards Understanding Sycophancy in Language Models
Authors: name: Mrinank Sharma, name: Meg Tong, name: Tomasz Korbak, name: David Duvenaud, name: Amanda Askell, name: Samuel R. Bowman, name: Newton Cheng, name: Esin Durmus, name: Zac Hatfield-Dodds, name: Scott R. Johnston, name: Shauna Kravec, name: Timothy Maxwell, name: Sam McCandlish, name: Kamal Ndousse, name: Oliver Rausch, name: Nicholas Schiefer, name: Da Yan, name: Miranda Zhang, name: Ethan Perez
Published: 2023-10-20 14:46:48

Reinforcement learning from human feedback (RLHF) is a popular technique for
training high-quality AI assistants. However, RLHF may also encourage model
responses that match user beliefs over truthful responses, a behavior known as
sycophancy. We investigate the prevalence of sycophancy in RLHF-trained models
and whether human preference judgements are responsible. We first demonstrate
that five state-of-the-art AI assistants consistently exhibit sycophantic
behavior across four varied free-form text-generation tasks. To understand if
human preferences drive this broadly observed behavior of RLHF models, we
analyze existing human preference data. We find that when a response matches a
user's views, it is more likely to be preferred. Moreover, both humans and
preference models (PMs) prefer convincingly-written sycophantic responses over
correct ones a negligible fraction of the time. Optimizing model outputs
against PMs also sometimes sacrifices truthfulness in favor of sycophancy.
Overall, our results indicate that sycophancy is a general behavior of RLHF
models, likely driven in part by human preference judgements favoring
sycophantic responses.