Title: TeLLMate: Trusted Execution for Large Language Models at the Edge
Primary area: foundation or frontier models, including LLMs
Keywords: Large Language Model, Security and Privacy

Abstract: Large Language Models (LLMs) are being increasingly deployed on edge devices to support real-time inference, preserve user privacy, and enhance energy efficiency. However, on-device deployment creates a lucrative attack surface for adversaries to steal the model. 
Trusted Execution Environments (TEEs) provide hardware-isolated enclaves to safeguard model parameters. However, existing approaches either demand costly retraining from substantial architectural modifications or incur significant communication overhead by protecting parameters across all layers within the TEE. To address these limitations, we propose TeLLMate, which helps identify the critical layers of LLMs for TEE protection. TeLLMate delivers two key capabilities: (1) a methodology for identifying layers critical to model performance using mathematical analysis. (2) a selective protection algorithm that minimizes the secure memory footprint while defending against retraining attacks. Our experimental results demonstrate that TeLLMate offers strong protection guarantees, resulting in at least a 15x increase in perplexity and close to random-guess accuracy in zero-shot downstream tasks for the attacker’s replicated model by including at most 10% of the LLM's parameters.

TL;DR: A secure and efficient framework that identifies and selectively protects the most critical layers of an LLM within a TEE.