On the Transformations across Reward Model, Parameter Update, and
  In-Context Prompt
Authors: name: Deng Cai; name: Huayang Li; name: Tingchen Fu; name: Siheng Li; name: Weiwen Xu; name: Shuaiyi Li; name: Bowen Cao; name: Zhisong Zhang; name: Xinting Huang; name: Leyang Cui; name: Yan Wang; name: Lemao Liu; name: Taro Watanabe; name: Shuming Shi
Published: 2024-06-24 07:42:32

Abstract: Despite the general capabilities of pre-trained large language models (LLMs),
they still need further adaptation to better serve practical applications. In
this paper, we demonstrate the interchangeability of three popular and distinct
adaptation tools: parameter updating, reward modeling, and in-context
prompting. This interchangeability establishes a triangular framework with six
transformation directions, each of which facilitates a variety of applications.
Our work offers a holistic view that unifies numerous existing studies and
suggests potential research directions. We envision our work as a useful
roadmap for future research on LLMs.