Coffee-Gym: An Environment for Evaluating and Improving Natural Language
  Feedback on Erroneous Code
Authors: name: Hyungjoo Chae; name: Taeyoon Kwon; name: Seungjun Moon; name: Yongho Song; name: Dongjin Kang; name: Kai Tzu-iunn Ong; name: Beong-woo Kwak; name: Seonghyeon Bae; name: Seung-won Hwang; name: Jinyoung Yeo
Published: 2024-09-29 14:14:25

Abstract: This paper presents Coffee-Gym, a comprehensive RL environment for training
models that provide feedback on code editing. Coffee-Gym includes two major
components: (1) Coffee, a dataset containing humans' code edit traces for
coding questions and machine-written feedback for editing erroneous code; (2)
CoffeeEval, a reward function that faithfully reflects the helpfulness of
feedback by assessing the performance of the revised code in unit tests. With
them, Coffee-Gym addresses the unavailability of high-quality datasets for
training feedback models with RL, and provides more accurate rewards than the
SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback
models that outperform baselines in enhancing open-source code LLMs' code
editing, making them comparable with closed-source LLMs. We make the dataset
and the model checkpoint publicly available.