ReGAL: Refactoring Programs to Discover Generalizable Abstractions
Authors: name: Elias Stengel-Eskin; name: Archiki Prasad; name: Mohit Bansal
Published: 2024-01-29 18:45:30

Abstract: While large language models (LLMs) are increasingly being used for program
synthesis, they lack the global view needed to develop useful abstractions;
they generally predict programs one at a time, often repeating the same
functionality. Generating redundant code from scratch is both inefficient and
error-prone. To address this, we propose Refactoring for Generalizable
Abstraction Learning (ReGAL), a gradient-free method for learning a library of
reusable functions via code refactorization, i.e. restructuring code without
changing its execution output. ReGAL learns from a small set of existing
programs, iteratively verifying and refining its abstractions via execution. We
find that the shared function libraries discovered by ReGAL make programs
easier to predict across diverse domains. On three datasets (LOGO graphics
generation, Date reasoning, and TextCraft, a Minecraft-based text game), both
open-source and proprietary LLMs improve in accuracy when predicting programs
with ReGAL functions. For CodeLlama-13B, ReGAL results in absolute accuracy
increases of 11.5% on graphics, 26.1% on date understanding, and 8.1% on
TextCraft, outperforming GPT-3.5 in two of three domains. Our analysis reveals
ReGAL's abstractions encapsulate frequently-used subroutines as well as
environment dynamics.