Towards Building the Federated GPT: Federated Instruction Tuning
Authors: name: Jianyi Zhang; name: Saeed Vahidian; name: Martin Kuo; name: Chunyuan Li; name: Ruiyi Zhang; name: Guoyin Wang; name: Yiran Chen
Published: 2023-05-09 17:42:34

Abstract: While ``instruction-tuned" generative large language models (LLMs) have
demonstrated an impressive ability to generalize to new tasks, the training
phases heavily rely on large amounts of diverse and high-quality instruction
data (such as ChatGPT and GPT-4). Unfortunately, acquiring high-quality data,
especially when it comes to human-written data, can pose significant challenges
both in terms of cost and accessibility. Moreover, concerns related to privacy
can further limit access to such data, making the process of obtaining it a
complex and nuanced undertaking. Consequently, this hinders the generality of
the tuned models and may restrict their effectiveness in certain contexts. To
tackle this issue, our study introduces a new approach called Federated
Instruction Tuning (FedIT), which leverages federated learning (FL) as the
learning framework for the instruction tuning of LLMs. This marks the first
exploration of FL-based instruction tuning for LLMs. This is especially
important since text data is predominantly generated by end users. Therefore,
it is imperative to design and adapt FL approaches to effectively leverage
these users' diverse instructions stored on local devices, while preserving
privacy and ensuring data security. In the current paper, by conducting widely
used GPT-4 auto-evaluation, we demonstrate that by exploiting the heterogeneous
and diverse sets of instructions on the client's end with the proposed
framework FedIT, we improved the performance of LLMs compared to centralized
training with only limited local instructions. Further, in this paper, we
developed a Github repository named Shepherd. This repository offers a
foundational framework for exploring federated fine-tuning of LLMs using
heterogeneous instructions across diverse categories.