Orionnnn/hub-stats huggingface dataset:
TravelPlanner: A Benchmark for Real-World Planning with Language Agents
Authors: name: Jian Xie, name: Kai Zhang, name: Jiangjie Chen, name: Tinghui Zhu, name: Renze Lou, name: Yuandong Tian, name: Yanghua Xiao, name: Yu Su
Published: 2024-02-02 18:39:51

Planning has been part of the core pursuit for artificial intelligence since
its conception, but earlier AI agents mostly focused on constrained settings
because many of the cognitive substrates necessary for human-level planning
have been lacking. Recently, language agents powered by large language models
(LLMs) have shown interesting capabilities such as tool use and reasoning. Are
these language agents capable of planning in more complex settings that are out
of the reach of prior AI agents? To advance this investigation, we propose
TravelPlanner, a new planning benchmark that focuses on travel planning, a
common real-world planning scenario. It provides a rich sandbox environment,
various tools for accessing nearly four million data records, and 1,225
meticulously curated planning intents and reference plans. Comprehensive
evaluations show that the current language agents are not yet capable of
handling such complex planning tasks-even GPT-4 only achieves a success rate of
0.6%. Language agents struggle to stay on task, use the right tools to collect
information, or keep track of multiple constraints. However, we note that the
mere possibility for language agents to tackle such a complex problem is in
itself non-trivial progress. TravelPlanner provides a challenging yet
meaningful testbed for future language agents.
K-Level Reasoning with Large Language Models
Authors: name: Yadong Zhang, name: Shaoguang Mao, name: Tao Ge, name: Xun Wang, name: Yan Xia, name: Man Lan, name: Furu Wei
Published: 2024-02-02 16:07:05

While Large Language Models (LLMs) have demonstrated their proficiency in
complex reasoning tasks, their performance in dynamic, interactive, and
competitive scenarios - such as business strategy and stock market analysis -
remains underexplored. To bridge this gap, we formally explore the dynamic
reasoning capabilities of LLMs for decision-making in rapidly evolving
environments. We introduce two game theory-based pilot challenges that mirror
the complexities of real-world dynamic decision-making. These challenges are
well-defined, enabling clear, controllable, and precise evaluation of LLMs'
dynamic reasoning abilities. Through extensive experiments, we find that
existing reasoning methods tend to falter in dynamic settings that require
k-level thinking - a key concept not tackled by previous works. To address
this, we propose a novel reasoning approach for LLMs, named "K-Level
Reasoning". This approach adopts the perspective of rivals to recursively
employ k-level thinking based on available historical information, which
significantly improves the prediction accuracy of rivals' subsequent moves and
informs more strategic decision-making. This research not only sets a robust
quantitative benchmark for the assessment of dynamic reasoning but also
markedly enhances the proficiency of LLMs in dynamic contexts.
StepCoder: Improve Code Generation with Reinforcement Learning from
  Compiler Feedback
Authors: name: Shihan Dou, name: Yan Liu, name: Haoxiang Jia, name: Limao Xiong, name: Enyu Zhou, name: Junjie Shan, name: Caishuang Huang, name: Wei Shen, name: Xiaoran Fan, name: Zhiheng Xi, name: Yuhao Zhou, name: Tao Ji, name: Rui Zheng, name: Qi Zhang, name: Xuanjing Huang, name: Tao Gui
Published: 2024-02-02 13:14:31

The advancement of large language models (LLMs) has significantly propelled
the field of code generation. Previous work integrated reinforcement learning
(RL) with compiler feedback for exploring the output space of LLMs to enhance
code generation quality. However, the lengthy code generated by LLMs in
response to complex human requirements makes RL exploration a challenge. Also,
since the unit tests may not cover the complicated code, optimizing LLMs by
using these unexecuted code snippets is ineffective. To tackle these
challenges, we introduce StepCoder, a novel RL framework for code generation,
consisting of two main components: CCCS addresses the exploration challenge by
breaking the long sequences code generation task into a Curriculum of Code
Completion Subtasks, while FGO only optimizes the model by masking the
unexecuted code segments to provide Fine-Grained Optimization. In addition, we
furthermore construct the APPS+ dataset for RL training, which is manually
verified to ensure the correctness of unit tests. Experimental results show
that our method improves the ability to explore the output space and
outperforms state-of-the-art approaches in corresponding benchmarks.
PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large
  Language Models
Authors: name: Sihao Hu, name: Tiansheng Huang, name: Ling Liu
Published: 2024-02-02 03:22:12

We introduce Pok\'eLLMon, the first LLM-embodied agent that achieves
human-parity performance in tactical battle games, as demonstrated in Pok\'emon
battles. The design of Pok\'eLLMon incorporates three key strategies:
(i) In-context reinforcement learning that instantly consumes text-based
feedback derived from battles to iteratively refine the policy; (ii)
Knowledge-augmented generation that retrieves external knowledge to counteract
hallucination and enables the agent to act timely and properly; (iii)
Consistent action generation to mitigate the panic switching
phenomenon when the agent faces a powerful opponent and wants to elude the
battle. We show that online battles against human demonstrates
Pok\'eLLMon's human-like battle strategies and just-in-time decision
making, achieving 49\% of win rate in the Ladder competitions and 56\% of win
rate in the invited battles. Our implementation and playable battle logs are
available at: https://github.com/git-disl/PokeLLMon.
Specialized Language Models with Cheap Inference from Limited Domain
  Data
Authors: name: David Grangier, name: Angelos Katharopoulos, name: Pierre Ablin, name: Awni Hannun
Published: 2024-02-02 01:45:18

Large language models have emerged as a versatile tool but are challenging to
apply to tasks lacking large inference budgets and large in-domain training
sets. This work formalizes these constraints and distinguishes four important
variables: the pretraining budget (for training before the target domain is
known), the specialization budget (for training after the target domain is
known), the inference budget, and the in-domain training set size. Across these
settings, we compare different approaches from the machine learning literature.
Limited by inference cost, we find better alternatives to the standard practice
of training very large vanilla transformer models. In particular, we show that
hyper-networks and mixture of experts have better perplexity for large
pretraining budgets, while small models trained on importance sampled datasets
are attractive for large specialization budgets.
Repeat After Me: Transformers are Better than State Space Models at
  Copying
Authors: name: Samy Jelassi, name: David Brandfonbrener, name: Sham M. Kakade, name: Eran Malach
Published: 2024-02-01 21:44:11

Transformers are the dominant architecture for sequence modeling, but there
is growing interest in models that use a fixed-size latent state that does not
depend on the sequence length, which we refer to as "generalized state space
models" (GSSMs). In this paper we show that while GSSMs are promising in terms
of inference-time efficiency, they are limited compared to transformer models
on tasks that require copying from the input context. We start with a
theoretical analysis of the simple task of string copying and prove that a two
layer transformer can copy strings of exponential length while GSSMs are
fundamentally limited by their fixed-size latent state. Empirically, we find
that transformers outperform GSSMs in terms of efficiency and generalization on
synthetic tasks that require copying the context. Finally, we evaluate
pretrained large language models and find that transformer models dramatically
outperform state space models at copying and retrieving information from
context. Taken together, these results suggest a fundamental gap between
transformers and GSSMs on tasks of practical interest.
EVA-GAN: Enhanced Various Audio Generation via Scalable Generative
  Adversarial Networks
Authors: name: Shijia Liao, name: Shiyi Lan, name: Arun George Zachariah
Published: 2024-01-31 03:31:03

The advent of Large Models marks a new era in machine learning, significantly
outperforming smaller models by leveraging vast datasets to capture and
synthesize complex patterns. Despite these advancements, the exploration into
scaling, especially in the audio generation domain, remains limited, with
previous efforts didn't extend into the high-fidelity (HiFi) 44.1kHz domain and
suffering from both spectral discontinuities and blurriness in the
high-frequency domain, alongside a lack of robustness against out-of-domain
data. These limitations restrict the applicability of models to diverse use
cases, including music and singing generation. Our work introduces Enhanced
Various Audio Generation via Scalable Generative Adversarial Networks
(EVA-GAN), yields significant improvements over previous state-of-the-art in
spectral and high-frequency reconstruction and robustness in out-of-domain data
performance, enabling the generation of HiFi audios by employing an extensive
dataset of 36,000 hours of 44.1kHz audio, a context-aware module, a
Human-In-The-Loop artifact measurement toolkit, and expands the model to
approximately 200 million parameters. Demonstrations of our work are available
at https://double-blind-eva-gan.cc.
Machine Unlearning for Image-to-Image Generative Models
Authors: name: Guihong Li, name: Hsiang Hsu, name: Chun-Fu, name: Chen, name: Radu Marculescu
Published: 2024-02-01 05:35:25

Machine unlearning has emerged as a new paradigm to deliberately forget data
samples from a given model in order to adhere to stringent regulations.
However, existing machine unlearning methods have been primarily focused on
classification models, leaving the landscape of unlearning for generative
models relatively unexplored. This paper serves as a bridge, addressing the gap
by providing a unifying framework of machine unlearning for image-to-image
generative models. Within this framework, we propose a
computationally-efficient algorithm, underpinned by rigorous theoretical
analysis, that demonstrates negligible performance degradation on the retain
samples, while effectively removing the information from the forget samples.
Empirical studies on two large-scale datasets, ImageNet-1K and Places-365,
further show that our algorithm does not rely on the availability of the retain
samples, which further complies with data retention policy. To our best
knowledge, this work is the first that represents systemic, theoretical,
empirical explorations of machine unlearning specifically tailored for
image-to-image generative models. Our code is available at
https://github.com/jpmorganchase/l2l-generator-unlearning.
AToM: Amortized Text-to-Mesh using 2D Diffusion
Authors: name: Guocheng Qian, name: Junli Cao, name: Aliaksandr Siarohin, name: Yash Kant, name: Chaoyang Wang, name: Michael Vasilkovsky, name: Hsin-Ying Lee, name: Yuwei Fang, name: Ivan Skorokhodov, name: Peiye Zhuang, name: Igor Gilitschenski, name: Jian Ren, name: Bernard Ghanem, name: Kfir Aberman, name: Sergey Tulyakov
Published: 2024-02-01 18:59:56

We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh
framework optimized across multiple text prompts simultaneously. In contrast to
existing text-to-3D methods that often entail time-consuming per-prompt
optimization and commonly output representations other than polygonal meshes,
AToM directly generates high-quality textured meshes in less than 1 second with
around 10 times reduction in the training cost, and generalizes to unseen
prompts. Our key idea is a novel triplane-based text-to-mesh architecture with
a two-stage amortized optimization strategy that ensures stable training and
enables scalability. Through extensive experiments on various prompt
benchmarks, AToM significantly outperforms state-of-the-art amortized
approaches with over 4 times higher accuracy (in DF415 dataset) and produces
more distinguishable and higher-quality 3D outputs. AToM demonstrates strong
generalizability, offering finegrained 3D assets for unseen interpolated
prompts without further optimization during inference, unlike per-prompt
solutions.
Can Large Language Models Understand Context?
Authors: name: Yilun Zhu, name: Joel Ruben Antony Moniz, name: Shruti Bhargava, name: Jiarui Lu, name: Dhivya Piraviperumal, name: Site Li, name: Yuan Zhang, name: Hong Yu, name: Bo-Hsiang Tseng
Published: 2024-02-01 18:55:29

Understanding context is key to understanding human language, an ability
which Large Language Models (LLMs) have been increasingly seen to demonstrate
to an impressive extent. However, though the evaluation of LLMs encompasses
various domains within the realm of Natural Language Processing, limited
attention has been paid to probing their linguistic capability of understanding
contextual features. This paper introduces a context understanding benchmark by
adapting existing datasets to suit the evaluation of generative models. This
benchmark comprises of four distinct tasks and nine datasets, all featuring
prompts designed to assess the models' ability to understand context. First, we
evaluate the performance of LLMs under the in-context learning pretraining
scenario. Experimental results indicate that pre-trained dense models struggle
with understanding more nuanced contextual features when compared to
state-of-the-art fine-tuned models. Second, as LLM compression holds growing
significance in both research and real-world applications, we assess the
context understanding of quantized models under in-context-learning settings.
We find that 3-bit post-training quantization leads to varying degrees of
performance reduction on our benchmark. We conduct an extensive analysis of
these scenarios to substantiate our experimental results.
SymbolicAI: A framework for logic-based approaches combining generative
  models and solvers
Authors: name: Marius-Constantin Dinu, name: Claudiu Leoveanu-Condrei, name: Markus Holzleitner, name: Werner Zellinger, name: Sepp Hochreiter
Published: 2024-02-01 18:50:50

We introduce SymbolicAI, a versatile and modular framework employing a
logic-based approach to concept learning and flow management in generative
processes. SymbolicAI enables the seamless integration of generative models
with a diverse range of solvers by treating large language models (LLMs) as
semantic parsers that execute tasks based on both natural and formal language
instructions, thus bridging the gap between symbolic reasoning and generative
AI. We leverage probabilistic programming principles to tackle complex tasks,
and utilize differentiable and classical programming paradigms with their
respective strengths. The framework introduces a set of polymorphic,
compositional, and self-referential operations for data stream manipulation,
aligning LLM outputs with user objectives. As a result, we can transition
between the capabilities of various foundation models endowed with zero- and
few-shot learning capabilities and specialized, fine-tuned models or solvers
proficient in addressing specific problems. In turn, the framework facilitates
the creation and evaluation of explainable computational graphs. We conclude by
introducing a quality measure and its empirical score for evaluating these
computational graphs, and propose a benchmark that compares various
state-of-the-art LLMs across a set of complex workflows. We refer to the
empirical score as the "Vector Embedding for Relational Trajectory Evaluation
through Cross-similarity", or VERTEX score for short. The framework codebase
and benchmark are linked below.
OLMo: Accelerating the Science of Language Models
Authors: name: Dirk Groeneveld, name: Iz Beltagy, name: Pete Walsh, name: Akshita Bhagia, name: Rodney Kinney, name: Oyvind Tafjord, name: Ananya Harsh Jha, name: Hamish Ivison, name: Ian Magnusson, name: Yizhong Wang, name: Shane Arora, name: David Atkinson, name: Russell Authur, name: Khyathi Raghavi Chandu, name: Arman Cohan, name: Jennifer Dumas, name: Yanai Elazar, name: Yuling Gu, name: Jack Hessel, name: Tushar Khot, name: William Merrill, name: Jacob Morrison, name: Niklas Muennighoff, name: Aakanksha Naik, name: Crystal Nam, name: Matthew E. Peters, name: Valentina Pyatkin, name: Abhilasha Ravichander, name: Dustin Schwenk, name: Saurabh Shah, name: Will Smith, name: Emma Strubell, name: Nishant Subramani, name: Mitchell Wortsman, name: Pradeep Dasigi, name: Nathan Lambert, name: Kyle Richardson, name: Luke Zettlemoyer, name: Jesse Dodge, name: Kyle Lo, name: Luca Soldaini, name: Noah A. Smith, name: Hannaneh Hajishirzi
Published: 2024-02-01 18:28:55

Language models (LMs) have become ubiquitous in both NLP research and in
commercial product offerings. As their commercial importance has surged, the
most powerful models have become closed off, gated behind proprietary
interfaces, with important details of their training data, architectures, and
development undisclosed. Given the importance of these details in
scientifically studying these models, including their biases and potential
risks, we believe it is essential for the research community to have access to
powerful, truly open LMs. To this end, this technical report details the first
release of OLMo, a state-of-the-art, truly Open Language Model and its
framework to build and study the science of language modeling. Unlike most
prior efforts that have only released model weights and inference code, we
release OLMo and the whole framework, including training data and training and
evaluation code. We hope this release will empower and strengthen the open
research community and inspire a new wave of innovation.
CroissantLLM: A Truly Bilingual French-English Language Model
Authors: name: Manuel Faysse, name: Patrick Fernandes, name: Nuno Guerreiro, name: António Loison, name: Duarte Alves, name: Caio Corro, name: Nicolas Boizard, name: João Alves, name: Ricardo Rei, name: Pedro Martins, name: Antoni Bigata Casademunt, name: François Yvon, name: André Martins, name: Gautier Viaud, name: Céline Hudelot, name: Pierre Colombo
Published: 2024-02-01 17:17:55

We introduce CroissantLLM, a 1.3B language model pretrained on a set of 3T
English and French tokens, to bring to the research and industrial community a
high-performance, fully open-sourced bilingual model that runs swiftly on
consumer-grade local hardware. To that end, we pioneer the approach of training
an intrinsically bilingual model with a 1:1 English-to-French pretraining data
ratio, a custom tokenizer, and bilingual finetuning datasets. We release the
training dataset, notably containing a French split with manually curated,
high-quality, and varied data sources. To assess performance outside of
English, we craft a novel benchmark, FrenchBench, consisting of an array of
classification and generation tasks, covering various orthogonal aspects of
model performance in the French Language. Additionally, rooted in transparency
and to foster further Large Language Model research, we release codebases, and
dozens of checkpoints across various model sizes, training data distributions,
and training steps, as well as fine-tuned Chat models, and strong translation
models. We evaluate our model through the FMTI framework, and validate 81 % of
the transparency criteria, far beyond the scores of even most open initiatives.
This work enriches the NLP landscape, breaking away from previous
English-centric work in order to strengthen our understanding of
multilinguality in language models.
Transforming and Combining Rewards for Aligning Large Language Models
Authors: name: Zihao Wang, name: Chirag Nagpal, name: Jonathan Berant, name: Jacob Eisenstein, name: Alex D'Amour, name: Sanmi Koyejo, name: Victor Veitch
Published: 2024-02-01 16:39:28

A common approach for aligning language models to human preferences is to
first learn a reward model from preference data, and then use this reward model
to update the language model. We study two closely related problems that arise
in this approach. First, any monotone transformation of the reward model
preserves preference ranking; is there a choice that is ``better'' than others?
Second, we often wish to align language models to multiple properties: how
should we combine multiple reward models? Using a probabilistic interpretation
of the alignment procedure, we identify a natural choice for transformation for
(the common case of) rewards learned from Bradley-Terry preference models. This
derived transformation has two important properties. First, it emphasizes
improving poorly-performing outputs, rather than outputs that already score
well. This mitigates both underfitting (where some prompts are not improved)
and reward hacking (where the model learns to exploit misspecification of the
reward model). Second, it enables principled aggregation of rewards by linking
summation to logical conjunction: the sum of transformed rewards corresponds to
the probability that the output is ``good'' in all measured properties, in a
sense we make precise. Experiments aligning language models to be both helpful
and harmless using RLHF show substantial improvements over the baseline
(non-transformed) approach.
EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit
  Large Language Models
Authors: name: Xuchen Pan, name: Yanxi Chen, name: Yaliang Li, name: Bolin Ding, name: Jingren Zhou
Published: 2024-02-01 11:39:04

This work introduces EE-Tuning, a lightweight and economical solution to
training/tuning early-exit large language models (LLMs). In contrast to the
common approach of full-parameter pre-training, EE-Tuning augments any
pre-trained (and possibly fine-tuned) standard LLM with additional early-exit
layers that are tuned in a parameter-efficient manner, which requires
significantly less computational resources and training data. Our
implementation of EE-Tuning achieves outstanding training efficiency via
extensive performance optimizations, as well as scalability due to its full
compatibility with 3D parallelism. Results of systematic experiments validate
the efficacy of EE-Tuning, confirming that effective early-exit LLM inference
can be achieved with a limited training budget. In hope of making early-exit
LLMs accessible to the community, we release the source code of our
implementation of EE-Tuning at https://github.com/pan-x-c/EE-LLM.
Efficient Exploration for LLMs
Authors: name: Vikranth Dwaracherla, name: Seyed Mohammad Asghari, name: Botao Hao, name: Benjamin Van Roy
Published: 2024-02-01 07:32:24

We present evidence of substantial benefit from efficient exploration in
gathering human feedback to improve large language models. In our experiments,
an agent sequentially generates queries while fitting a reward model to the
feedback received. Our best-performing agent generates queries using double
Thompson sampling, with uncertainty represented by an epistemic neural network.
Our results demonstrate that efficient exploration enables high levels of
performance with far fewer queries. Further, both uncertainty estimation and
the choice of exploration scheme play critical roles.