Diffusion Augmented Agents: A Framework for Efficient Exploration and
  Transfer Learning
Authors: name: Norman Di Palo; name: Leonard Hasenclever; name: Jan Humplik; name: Arunkumar Byravan
Published: 2024-07-30 13:01:31

Abstract: We introduce Diffusion Augmented Agents (DAAG), a novel framework that
leverages large language models, vision language models, and diffusion models
to improve sample efficiency and transfer learning in reinforcement learning
for embodied agents. DAAG hindsight relabels the agent's past experience by
using diffusion models to transform videos in a temporally and geometrically
consistent way to align with target instructions with a technique we call
Hindsight Experience Augmentation. A large language model orchestrates this
autonomous process without requiring human supervision, making it well-suited
for lifelong learning scenarios. The framework reduces the amount of
reward-labeled data needed to 1) finetune a vision language model that acts as
a reward detector, and 2) train RL agents on new tasks. We demonstrate the
sample efficiency gains of DAAG in simulated robotics environments involving
manipulation and navigation. Our results show that DAAG improves learning of
reward detectors, transferring past experience, and acquiring new tasks - key
abilities for developing efficient lifelong learning agents. Supplementary
material and visualizations are available on our website
https://sites.google.com/view/diffusion-augmented-agents/