Orionnnn/hub-stats huggingface dataset:
Improved Baselines with Visual Instruction Tuning
Authors: name: Haotian Liu, name: Chunyuan Li, name: Yuheng Li, name: Yong Jae Lee
Published: 2023-10-05 17:59:56

Large multimodal models (LMM) have recently shown encouraging progress with
visual instruction tuning. In this note, we show that the fully-connected
vision-language cross-modal connector in LLaVA is surprisingly powerful and
data-efficient. With simple modifications to LLaVA, namely, using
CLIP-ViT-L-336px with an MLP projection and adding academic-task-oriented VQA
data with simple response formatting prompts, we establish stronger baselines
that achieve state-of-the-art across 11 benchmarks. Our final 13B checkpoint
uses merely 1.2M publicly available data, and finishes full training in ~1 day
on a single 8-A100 node. We hope this can make state-of-the-art LMM research
more accessible. Code and model will be publicly available.
A Long Way to Go: Investigating Length Correlations in RLHF
Authors: name: Prasann Singhal, name: Tanya Goyal, name: Jiacheng Xu, name: Greg Durrett
Published: 2023-10-05 17:38:28

Great successes have been reported using Reinforcement Learning from Human
Feedback (RLHF) to align large language models. Open-source preference datasets
and reward models have enabled wider experimentation beyond generic chat
settings, particularly to make systems more "helpful" for tasks like web
question answering, summarization, and multi-turn dialogue. When optimizing for
helpfulness, RLHF has been consistently observed to drive models to produce
longer outputs. This paper demonstrates that optimizing for response length is
a significant factor behind RLHF's reported improvements in these settings.
First, we study the relationship between reward and length for reward models
trained on three open-source preference datasets for helpfulness. Here, length
correlates strongly with reward, and improvements in reward score are driven in
large part by shifting the distribution over output lengths. We then explore
interventions during both RL and reward model learning to see if we can achieve
the same downstream improvements as RLHF without increasing length. While our
interventions mitigate length increases, they aren't uniformly effective across
settings. Furthermore, we find that even running RLHF with a reward based
solely on length can reproduce most of the downstream improvements over the
initial policy model, showing that reward models in these settings have a long
way to go.
SteP: Stacked LLM Policies for Web Actions
Authors: name: Paloma Sodhi, name: S. R. K. Branavan, name: Ryan McDonald
Published: 2023-10-05 17:40:09

Performing tasks on the web presents fundamental challenges to large language
models (LLMs), including combinatorially large open-world tasks and variations
across web interfaces. Simply specifying a large prompt to handle all possible
behaviors and states is extremely complex, and results in behavior leaks
between unrelated behaviors. Decomposition to distinct policies can address
this challenge, but requires carefully handing off control between policies. We
propose Stacked LLM Policies for Web Actions (SteP), an approach to dynamically
compose policies to solve a diverse set of web tasks. SteP defines a Markov
Decision Process where the state is a stack of policies representing the
control state, i.e., the chain of policy calls. Unlike traditional methods that
are restricted to static hierarchies, SteP enables dynamic control that adapts
to the complexity of the task. We evaluate SteP against multiple baselines and
web environments including WebArena, MiniWoB++, and a CRM. On WebArena, SteP
improves (14.9\% to 33.5\%) over SOTA that use GPT-4 policies, while on
MiniWob++, SteP is competitive with prior works while using significantly less
data. Our code and data are available at
https://asappresearch.github.io/webagents-step.
MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical
  Reasoning
Authors: name: Ke Wang, name: Houxing Ren, name: Aojun Zhou, name: Zimu Lu, name: Sichun Luo, name: Weikang Shi, name: Renrui Zhang, name: Linqi Song, name: Mingjie Zhan, name: Hongsheng Li
Published: 2023-10-05 17:52:09

The recently released GPT-4 Code Interpreter has demonstrated remarkable
proficiency in solving challenging math problems, primarily attributed to its
ability to seamlessly reason with natural language, generate code, execute
code, and continue reasoning based on the execution output. In this paper, we
present a method to fine-tune open-source language models, enabling them to use
code for modeling and deriving math equations and, consequently, enhancing
their mathematical reasoning abilities. We propose a method of generating novel
and high-quality datasets with math problems and their code-based solutions,
referred to as MathCodeInstruct. Each solution interleaves natural language,
code, and execution results. We also introduce a customized supervised
fine-tuning and inference approach. This approach yields the MathCoder models,
a family of models capable of generating code-based solutions for solving
challenging math problems. Impressively, the MathCoder models achieve
state-of-the-art scores among open-source LLMs on the MATH (45.2%) and GSM8K
(83.9%) datasets, substantially outperforming other open-source alternatives.
Notably, the MathCoder model not only surpasses ChatGPT-3.5 and PaLM-2 on GSM8K
and MATH but also outperforms GPT-4 on the competition-level MATH dataset. The
dataset and models will be released at https://github.com/mathllm/MathCoder.
Aligning Text-to-Image Diffusion Models with Reward Backpropagation
Authors: name: Mihir Prabhudesai, name: Anirudh Goyal, name: Deepak Pathak, name: Katerina Fragkiadaki
Published: 2023-10-05 17:59:18

Text-to-image diffusion models have recently emerged at the forefront of
image generation, powered by very large-scale unsupervised or weakly supervised
text-to-image training datasets. Due to their unsupervised training,
controlling their behavior in downstream tasks, such as maximizing
human-perceived image quality, image-text alignment, or ethical image
generation, is difficult. Recent works finetune diffusion models to downstream
reward functions using vanilla reinforcement learning, notorious for the high
variance of the gradient estimators. In this paper, we propose AlignProp, a
method that aligns diffusion models to downstream reward functions using
end-to-end backpropagation of the reward gradient through the denoising
process. While naive implementation of such backpropagation would require
prohibitive memory resources for storing the partial derivatives of modern
text-to-image models, AlignProp finetunes low-rank adapter weight modules and
uses gradient checkpointing, to render its memory usage viable. We test
AlignProp in finetuning diffusion models to various objectives, such as
image-text semantic alignment, aesthetics, compressibility and controllability
of the number of objects present, as well as their combinations. We show
AlignProp achieves higher rewards in fewer training steps than alternatives,
while being conceptually simpler, making it a straightforward choice for
optimizing diffusion models for differentiable reward functions of interest.
Code and Visualization results are available at https://align-prop.github.io/.
Leveraging Unpaired Data for Vision-Language Generative Models via Cycle
  Consistency
Authors: name: Tianhong Li, name: Sangnie Bhardwaj, name: Yonglong Tian, name: Han Zhang, name: Jarred Barber, name: Dina Katabi, name: Guillaume Lajoie, name: Huiwen Chang, name: Dilip Krishnan
Published: 2023-10-05 17:55:19

Current vision-language generative models rely on expansive corpora of paired
image-text data to attain optimal performance and generalization capabilities.
However, automatically collecting such data (e.g. via large-scale web scraping)
leads to low quality and poor image-text correlation, while human annotation is
more accurate but requires significant manual effort and expense. We introduce
ITIT (InTegrating Image
Text): an innovative training paradigm grounded in the concept of
cycle consistency which allows vision-language training on unpaired image and
text data. ITIT is comprised of a joint image-text encoder with disjoint image
and text decoders that enable bidirectional image-to-text and text-to-image
generation in a single framework. During training, ITIT leverages a small set
of paired image-text data to ensure its output matches the input reasonably
well in both directions. Simultaneously, the model is also trained on much
larger datasets containing only images or texts. This is achieved by enforcing
cycle consistency between the original unpaired samples and the cycle-generated
counterparts. For instance, it generates a caption for a given input image and
then uses the caption to create an output image, and enforces similarity
between the input and output images. Our experiments show that ITIT with
unpaired datasets exhibits similar scaling behavior as using high-quality
paired data. We demonstrate image generation and captioning performance on par
with state-of-the-art text-to-image and image-to-text models with orders of
magnitude fewer (only 3M) paired image-text data.
FreshLLMs: Refreshing Large Language Models with Search Engine
  Augmentation
Authors: name: Tu Vu, name: Mohit Iyyer, name: Xuezhi Wang, name: Noah Constant, name: Jerry Wei, name: Jason Wei, name: Chris Tar, name: Yun-Hsuan Sung, name: Denny Zhou, name: Quoc Le, name: Thang Luong
Published: 2023-10-05 00:04:12

Most large language models (LLMs) are trained once and never updated; thus,
they lack the ability to dynamically adapt to our ever-changing world. In this
work, we perform a detailed study of the factuality of LLM-generated text in
the context of answering questions that test current world knowledge.
Specifically, we introduce FreshQA, a novel dynamic QA benchmark encompassing a
diverse range of question and answer types, including questions that require
fast-changing world knowledge as well as questions with false premises that
need to be debunked. We benchmark a diverse array of both closed and
open-source LLMs under a two-mode evaluation procedure that allows us to
measure both correctness and hallucination. Through human evaluations involving
more than 50K judgments, we shed light on limitations of these models and
demonstrate significant room for improvement: for instance, all models
(regardless of model size) struggle on questions that involve fast-changing
knowledge and false premises. Motivated by these results, we present
FreshPrompt, a simple few-shot prompting method that substantially boosts the
performance of an LLM on FreshQA by incorporating relevant and up-to-date
information retrieved from a search engine into the prompt. Our experiments
show that FreshPrompt outperforms both competing search engine-augmented
prompting methods such as Self-Ask (Press et al., 2022) as well as commercial
systems such as Perplexity.AI. Further analysis of FreshPrompt reveals that
both the number of retrieved evidences and their order play a key role in
influencing the correctness of LLM-generated answers. Additionally, instructing
the LLM to generate concise and direct answers helps reduce hallucination
compared to encouraging more verbose answers. To facilitate future work, we
release FreshQA at github.com/freshllms/freshqa and commit to updating it at
regular intervals.
Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and
  Latent Diffusion
Authors: name: Anton Razzhigaev, name: Arseniy Shakhmatov, name: Anastasia Maltseva, name: Vladimir Arkhipkin, name: Igor Pavlov, name: Ilya Ryabov, name: Angelina Kuts, name: Alexander Panchenko, name: Andrey Kuznetsov, name: Denis Dimitrov
Published: 2023-10-05 12:29:41

Text-to-image generation is a significant domain in modern computer vision
and has achieved substantial improvements through the evolution of generative
architectures. Among these, there are diffusion-based models that have
demonstrated essential quality enhancements. These models are generally split
into two categories: pixel-level and latent-level approaches. We present
Kandinsky1, a novel exploration of latent diffusion architecture, combining the
principles of the image prior models with latent diffusion techniques. The
image prior model is trained separately to map text embeddings to image
embeddings of CLIP. Another distinct feature of the proposed model is the
modified MoVQ implementation, which serves as the image autoencoder component.
Overall, the designed model contains 3.3B parameters. We also deployed a
user-friendly demo system that supports diverse generative modes such as
text-to-image generation, image fusion, text and image fusion, image variations
generation, and text-guided inpainting/outpainting. Additionally, we released
the source code and checkpoints for the Kandinsky models. Experimental
evaluations demonstrate a FID score of 8.03 on the COCO-30K dataset, marking
our model as the top open-source performer in terms of measurable image
generation quality.
DSPy: Compiling Declarative Language Model Calls into Self-Improving
  Pipelines
Authors: name: Omar Khattab, name: Arnav Singhvi, name: Paridhi Maheshwari, name: Zhiyuan Zhang, name: Keshav Santhanam, name: Sri Vardhamanan, name: Saiful Haq, name: Ashutosh Sharma, name: Thomas T. Joshi, name: Hanna Moazam, name: Heather Miller, name: Matei Zaharia, name: Christopher Potts
Published: 2023-10-05 17:37:25

The ML community is rapidly exploring techniques for prompting language
models (LMs) and for stacking them into pipelines that solve complex tasks.
Unfortunately, existing LM pipelines are typically implemented using hard-coded
"prompt templates", i.e. lengthy strings discovered via trial and error. Toward
a more systematic approach for developing and optimizing LM pipelines, we
introduce DSPy, a programming model that abstracts LM pipelines as text
transformation graphs, i.e. imperative computational graphs where LMs are
invoked through declarative modules. DSPy modules are parameterized, meaning
they can learn (by creating and collecting demonstrations) how to apply
compositions of prompting, finetuning, augmentation, and reasoning techniques.
We design a compiler that will optimize any DSPy pipeline to maximize a given
metric. We conduct two case studies, showing that succinct DSPy programs can
express and optimize sophisticated LM pipelines that reason about math word
problems, tackle multi-hop retrieval, answer complex questions, and control
agent loops. Within minutes of compiling, a few lines of DSPy allow GPT-3.5 and
llama2-13b-chat to self-bootstrap pipelines that outperform standard few-shot
prompting (generally by over 25% and 65%, respectively) and pipelines with
expert-created demonstrations (by up to 5-46% and 16-40%, respectively). On top
of that, DSPy programs compiled to open and relatively small LMs like
770M-parameter T5 and llama2-13b-chat are competitive with approaches that rely
on expert-written prompt chains for proprietary GPT-3.5. DSPy is available at
https://github.com/stanfordnlp/dspy
Large Language Model Cascades with Mixture of Thoughts Representations
  for Cost-efficient Reasoning
Authors: name: Murong Yue, name: Jie Zhao, name: Min Zhang, name: Liang Du, name: Ziyu Yao
Published: 2023-10-04 18:21:17

Large language models (LLMs) such as GPT-4 have exhibited remarkable
performance in a variety of tasks, but this strong performance often comes with
the high expense of using paid API services. In this paper, we are motivated to
study building an LLM cascade to save the cost of using LLMs, particularly for
performing reasoning (e.g., mathematical, causal) tasks. Our cascade pipeline
follows the intuition that simpler questions can be addressed by a weaker but
more affordable LLM, whereas only the challenging questions necessitate the
stronger and more expensive LLM. To realize this decision-making, we consider
the "answer consistency" of the weaker LLM as a signal of the question
difficulty and propose several methods for the answer sampling and consistency
checking, including one leveraging a mixture of two thought representations
(i.e., Chain-of-Thought and Program-of-Thought). Through experiments on six
reasoning benchmark datasets, with GPT-3.5-turbo and GPT-4 being the weaker and
stronger LLMs, respectively, we demonstrate that our proposed LLM cascades can
achieve performance comparable to using solely the stronger LLM but require
only 40% of its cost.
How FaR Are Large Language Models From Agents with Theory-of-Mind?
Authors: name: Pei Zhou, name: Aman Madaan, name: Srividya Pranavi Potharaju, name: Aditya Gupta, name: Kevin R. McKee, name: Ari Holtzman, name: Jay Pujara, name: Xiang Ren, name: Swaroop Mishra, name: Aida Nematzadeh, name: Shyam Upadhyay, name: Manaal Faruqui
Published: 2023-10-04 06:47:58

"Thinking is for Doing." Humans can infer other people's mental states from
observations--an ability called Theory-of-Mind (ToM)--and subsequently act
pragmatically on those inferences. Existing question answering benchmarks such
as ToMi ask models questions to make inferences about beliefs of characters in
a story, but do not test whether models can then use these inferences to guide
their actions. We propose a new evaluation paradigm for large language models
(LLMs): Thinking for Doing (T4D), which requires models to connect inferences
about others' mental states to actions in social scenarios. Experiments on T4D
demonstrate that LLMs such as GPT-4 and PaLM 2 seemingly excel at tracking
characters' beliefs in stories, but they struggle to translate this capability
into strategic action. Our analysis reveals the core challenge for LLMs lies in
identifying the implicit inferences about mental states without being
explicitly asked about as in ToMi, that lead to choosing the correct action in
T4D. To bridge this gap, we introduce a zero-shot prompting framework, Foresee
and Reflect (FaR), which provides a reasoning structure that encourages LLMs to
anticipate future challenges and reason about potential actions. FaR boosts
GPT-4's performance from 50% to 71% on T4D, outperforming other prompting
methods such as Chain-of-Thought and Self-Ask. Moreover, FaR generalizes to
diverse out-of-distribution story structures and scenarios that also require
ToM inferences to choose an action, consistently outperforming other methods
including few-shot in-context learning.
EcoAssistant: Using LLM Assistant More Affordably and Accurately
Authors: name: Jieyu Zhang, name: Ranjay Krishna, name: Ahmed H. Awadallah, name: Chi Wang
Published: 2023-10-03 22:16:13

Today, users ask Large language models (LLMs) as assistants to answer queries
that require external knowledge; they ask about the weather in a specific city,
about stock prices, and even about where specific locations are within their
neighborhood. These queries require the LLM to produce code that invokes
external APIs to answer the user's question, yet LLMs rarely produce correct
code on the first try, requiring iterative code refinement upon execution
results. In addition, using LLM assistants to support high query volumes can be
expensive. In this work, we contribute a framework, EcoAssistant, that enables
LLMs to answer code-driven queries more affordably and accurately. EcoAssistant
contains three components. First, it allows the LLM assistants to converse with
an automatic code executor to iteratively refine code or to produce answers
based on the execution results. Second, we use a hierarchy of LLM assistants,
which attempts to answer the query with weaker, cheaper LLMs before backing off
to stronger, expensive ones. Third, we retrieve solutions from past successful
queries as in-context demonstrations to help subsequent queries. Empirically,
we show that EcoAssistant offers distinct advantages for affordability and
accuracy, surpassing GPT-4 by 10 points of success rate with less than 50% of
GPT-4's cost.
Large Language Models Cannot Self-Correct Reasoning Yet
Authors: name: Jie Huang, name: Xinyun Chen, name: Swaroop Mishra, name: Huaixiu Steven Zheng, name: Adams Wei Yu, name: Xinying Song, name: Denny Zhou
Published: 2023-10-03 04:56:12

Large Language Models (LLMs) have emerged as a groundbreaking technology with
their unparalleled text generation capabilities across various applications.
Nevertheless, concerns persist regarding the accuracy and appropriateness of
their generated content. A contemporary methodology, self-correction, has been
proposed as a remedy to these issues. Building upon this premise, this paper
critically examines the role and efficacy of self-correction within LLMs,
shedding light on its true potential and limitations. Central to our
investigation is the notion of intrinsic self-correction, whereby an LLM
attempts to correct its initial responses based solely on its inherent
capabilities, without the crutch of external feedback. In the context of
reasoning, our research indicates that LLMs struggle to self-correct their
responses without external feedback, and at times, their performance might even
degrade post self-correction. Drawing from these insights, we offer suggestions
for future research and practical applications in this field.
ImagenHub: Standardizing the evaluation of conditional image generation
  models
Authors: name: Max Ku, name: Tianle Li, name: Kai Zhang, name: Yujie Lu, name: Xingyu Fu, name: Wenwen Zhuang, name: Wenhu Chen
Published: 2023-10-02 19:41:42

Recently, a myriad of conditional image generation and editing models have
been developed to serve different downstream tasks, including text-to-image
generation, text-guided image editing, subject-driven image generation,
control-guided image generation, etc. However, we observe huge inconsistencies
in experimental conditions: datasets, inference, and evaluation metrics -
render fair comparisons difficult. This paper proposes ImagenHub, which is a
one-stop library to standardize the inference and evaluation of all the
conditional image generation models. Firstly, we define seven prominent tasks
and curate high-quality evaluation datasets for them. Secondly, we built a
unified inference pipeline to ensure fair comparison. Thirdly, we design two
human evaluation scores, i.e. Semantic Consistency and Perceptual Quality,
along with comprehensive guidelines to evaluate generated images. We train
expert raters to evaluate the model outputs based on the proposed metrics. Our
human evaluation achieves a high inter-worker agreement of Krippendorff's alpha
on 76% models with a value higher than 0.4. We comprehensively evaluated a
total of around 30 models and observed three key takeaways: (1) the existing
models' performance is generally unsatisfying except for Text-guided Image
Generation and Subject-driven Image Generation, with 74% models achieving an
overall score lower than 0.5. (2) we examined the claims from published papers
and found 83% of them hold with a few exceptions. (3) None of the existing
automatic metrics has a Spearman's correlation higher than 0.2 except
subject-driven image generation. Moving forward, we will continue our efforts
to evaluate newly published models and update our leaderboard to keep track of
the progress in conditional image generation.