DORY : Deliberative Prompt Recovery for LLM (2024)

Abstract: Prompt recovery in large language models (LLMs) is crucial for understanding how LLMs work and addressing concerns regarding privacy, copyright, etc. The trend towards inference-only APIs complicates this task by restricting access to essential outputs for recovery. To tackle this challenge, we extract prompt-related information from limited outputs and identify a strong(negative) correlation between output probability-based uncertainty and the success of prompt recovery.This finding led to the development of Deliberative PrOmpt RecoverY (DORY), our novel approach that leverages uncertainty to recover prompts accurately. DORY involves reconstructing drafts from outputs, refining these with hints, and filtering out noise based on uncertainty. Our evaluation shows that DORY outperforms existing baselines across diverse LLMs and prompt benchmarks, improving performance by approximately 10.82% and establishing a new state-of-the-art record in prompt recovery tasks. Significantly, DORY operates using a single LLM without any external resources or model, offering a cost-effective, user-friendly prompt recovery solution.

## introduction large language models (llms) are widely applied for their groundbreaking performance across various tasks, typically by parsing user-inputted prompts to generate output text. considering scenarios where the input prompt is agnostic, it is particularly critical to recover the prompt from the output -dubbed as "prompt recovery". prompt recovery interests researchers for its ability to reveal system prompts of closed-source llms @xcite and detect security vulnerabilities, such as user's query theft @xcite , highquality prompt leaks @xcite , and defenses bypassing to generate prompt when did the american revolution begin? the american revolution began on april 19, 1775, with the battles of lexington and concord in massachusetts. the conflict eventually led to the formal declaration of independence from britain and the establishment of the united states of america. ## motivation in this section, we explore the feasibility of prompt recovery for api-based llms using limited output-the output text and output probabilities. ## method in this section, we introduce the deliberative prompt recovery (dory) framework, crafted to leverage output probability-based uncertainty to facilitate prompt recovery for llms independently. depicted in figure 4 , the main pathway of dory is to recover prompt from clues-a combination of outputs, draft, hint, and noise-consisting of three core components: ➀-draft reconstruction: reconstructing the draft from output text (sec 4.1); ➁-hint refinement: generating hint (i.e., shared tokens) based on uncertainty from outputs (sec 4.2); ➂-noise reduction: forming draft outputs from the draft, followed by generating the draft hint, then comparing draft hint and hint to separate the noise (i.e., non-shared tokens) (sec 4.3). ## experiments in this section, we present extensive experimental results and detailed analysis.