Federated Data-Efficient Instruction Tuning for Large Language Models (2025)

Abstract: Instruction tuning is a crucial step in improving the responsiveness of pretrained large language models (LLMs) to human instructions. Federated learning (FL) helps to exploit the use of vast private instruction data from clients, becoming popular for LLM tuning by improving data diversity. Existing federated tuning simply consumes all local data, causing excessive computational overhead and overfitting to local data, while centralized data-efficient solutions are not suitable for FL due to privacy concerns. This work presents FedHDS, a federated data-efficient instruction tuning approach, which tunes LLMs with a representative subset of edge-side data. It reduces the data redundancy at both intra- and inter-client levels without sharing raw data. Experiments with various LLMs, datasets and partitions show that FedHDS improves Rouge-L on unseen tasks by an average of 10.72% over the SOTA full-data federated instruction tuning methods, while using less than 1.5% of the data samples, improving training efficiency by up to tens of times.

## introduction large language models (llms) exhibit remarkable performance on a wide range of natural language tasks. instruction tuning @xcite is crucial to improve llms' responsiveness to human instructions, whose success hinges on the availability of diverse and high-quality data @xcite @xcite , particularly for unseen tasks @xcite . as the pool of publicly available data is expected to be exhausted in the near future @xcite , exploiting more data sources, such as data on edge devices, becomes essential for continued llm improvement @xcite . however, privacy concerns and regulations such as gdpr @xcite complicate the use of edge-side data. federated learning (fl) @xcite has emerged as a promising solution to utilize diverse data from edge devices to tune llms @xcite , especially for generalization to unseen tasks @xcite . prior fl works on llms mainly focus on communication and memory costs @xcite @xcite @xcite @xcite . although these methods show promising results, our investigation reveals that they often use all local data for training @xcite @xcite . this strategy leads to many training steps, and thus causes two main issues in fl contexts: despite their success in centralized scenarios, applying these works to fl faces two main limitations. (l1) low compatibility with fl: some methods access all data simultaneously @xcite @xcite , violating fl's fundamental principle of restricting direct access to data by third parties, and thus failing to detect inter-client data redundancy. (l2) suboptimal data representations: the selection of coresets significantly impacts the tuning accuracy @xcite . current methods rely on data representations from the last transformer layer @xcite @xcite , which may not capture the full spectrum of features to distinguish data samples. due to the limited research on federated dataefficient instruction tuning and the shortcomings of centralized methods in fl scenarios, we propose fedhds, a novel framework of federated hierarchical data selection. it clusters local data on each client to detect intra-client data redundancy, and then sends approximate cluster centroids to the server for further clustering to identify inter-client data redundancy, cutting down computational costs and enhancing generalization to new tasks by taking the distribution of edge-side data without direct data access, which is compatible with fl (l1). then, we propose to fuse data features from different transformer layers, in order to provide better data representations for coreset selection (l2). this work makes the following contributions: ## related work federated learning for llm tuning federated tuning for llms gains widespread attention by enabling edge-side data use without direct access. due to the scale of llms, many studies develop federated tuning based on parameter-efficient fine-tuning (peft) techniques to reduce memory and communication costs @xcite @xcite . among peft techniques, lora @xcite ) gains significant attention, with studies on initialization @xcite , objective consistency @xcite , and heterogeneous client resources @xcite . some works focus on specific costs, such as communication by transmitting only loss values and random seeds @xcite and memory by employing quantization-aware training @xcite . during local training, existing works typically consume all local data @xcite @xcite @xcite , leading to a significant computational cost and overfitting to the local data, as client-side data in fl often consists of many samples from only a few domains. unlike existing methods, this work takes a data-centric perspective @xcite , which focuses on data efficiency by selecting a small number of representative data samples for llm tuning, to achieve better efficiency and generalization on unseen tasks. ## problem formulation federated instruction tuning assuming there are n clients in an fl system, each client i holds a private dataset d i containing several instances of instruction data. given an llm @xmath0, federated instruction tuning aims at optimizing w with the private data held by clients towards the following objective, ## conclusion existing federated instruction tuning methods for llms typically train llms using all local data, causing significant computation cost and overfitting to local data. this work pioneers an exploration into federated data-efficient instruction tuning, and proposes fedhds, a coreset selection approach that solves both intra-and inter-client data redundancy. it fuses data features of varying abstraction levels obtained from different transformer layers for better data representation. extensive experiments involving various datasets, llms and non-iidness demonstrate that fedhds enhances the data efficiency and rouge-l on unseen tasks over existing federated tuning methods.