JAMDEC : Unsupervised Authorship Obfuscation using Constrained Decoding over Small Language Models (2024)

Abstract: The permanence of online content combined with the enhanced authorship identification techniques calls for stronger computational methods to protect the identity and privacy of online authorship when needed, e.g., blind reviews for scientific papers, anonymous online reviews, or anonymous interactions in the mental health forums. In this paper, we propose an unsupervised inference-time approach to authorship obfuscation to address the unique challenges of authorship obfuscation: lack of supervision data for diverse authorship and domains, and the need for a sufficient level of revision beyond simple paraphrasing to obfuscate the authorship, all the while preserving the original content and fluency.We introduce JAMDEC, a user-controlled, inference-time algorithm for authorship obfuscation that can be in principle applied to any text and authorship. Our approach builds on small language models such as GPT2-XL in order to help avoid disclosing the original content to proprietary LLM’s APIs, while also reducing the performance gap between small and large language models via algorithmic enhancement. The key idea behind our approach is to boost the creative power of smaller language models through constrained decoding, while also allowing for user-specified controls and flexibility. Experimental results demonstrate that our approach based on GPT2-XL outperforms previous state-of-the-art methods based on comparably small models, while performing competitively against GPT3.5 175B, a propriety model that is two orders of magnitudes larger.

## introduction authorship obfuscation, the task of rewriting a text to protect the original writer's identity, has become increasingly important given the permanence of online content combined with new enhanced authorship attribution techniques @xcite figure 1 : jamdec framework. @xcite . this task holds implications in various domains, including online privacy, and blind review in academic research. however, safeguarding an authorship style, while maintaining the same content and grammatical fluency, is a complex task. unlike other authorship-related tasks such as paraphrasing or style transfer, authorship obfuscation poses unique technical challenges due to its different assumptions. for example, paraphrasing involves rephrasing an original text, but can be accomplished without altering the original style. conversely, for style transfer, the task requires a predetermined target style. however, in the case of authorship obfuscation, there is no fixed endpoint style to guide the generation because the main goal is the absence or avoidance of a particular style. in fact, it may involve incorporating multiple styles or navigating a wide spectrum of possibilities to achieve success. 1 one approach to authorship obfuscation is to use large language models, such as chatgpt or gpt4. however, these models require large computing resources. furthermore, if a user employs a method based on proprietary llms that retain user data, they are vulnerable to extra privacy threats or the leakage of their original content. to mitigate these risks, non-model or smaller closed model methods are preferred. other previous approaches for authorship obfuscation include the use of round-trip machine translation @xcite , strict rule-based algorithms @xcite , or iterativechange algorithms @xcite . however, these methods either do not lead to enough modification @xcite , diverge into grammatically incorrect text due to the rigid rules @xcite , or require an additional large-scale authorship corpus @xcite . therefore, in comparison to modern llms, we find a notable performance gap between previous methods developed for smaller models. to overcome these limitations, we present jamdec, a light-weight, user-controlled, unsupervised inference time algorithm for authorship obfuscation that can be used with any arbitrary text. jamdec employs smaller base models such as gpt2, which by themselves are too weak to produce accurate paraphrases, let alone obfuscation @xcite . to overcome this weakness, we frame the task as a constraint decoding problem, where the constraint is given as lexical keywords to include to control the content of the generation. to identify these keywords automatically, we leverage likelihood scores from smaller models. lastly, since the decoded text is not guaranteed to be faithful to the original text, we design a filtering step that can be uniquely adjusted by the user. an overview of jamdec three-stage framework can be found in figure 1 . the name is inspired by jambalaya, the popular american creole and cajun rice dish which is a mixture of meat, vegetables and spices. we provide experimentation on two datasets, scholarly articles and diary-style entries with a range of three to ten authors. the results show that jamdec performs better than state-of-the-art meth-1 a more detailed discussion on the differences between these authorship-tasks can be found in appendix f. ods of similar size and comparable to significantly larger language models in both automatic and human evaluations. in particular, we demonstrate that jamdec is able to obfuscate, while simultaneously preserving the original content, which previous methods cannot achieve. 2 2 background on authorship obfuscation setup. let a be a given set of authors. we consider an original text y orig that was written by author @xmath0. the task of authorship obfuscation aims to create a new text y obf which can not be identified as written by author b. for evaluation, we consider a classification model m (•) (also known as an authorship attribution models), which has been trained to classify texts of each author in a. the aim is to create a method f (•) such that m (f (y orig )) ̸ = b. measure of a successful algorithm. our goal is to create an obfuscated version of the original text that preserves the meaning and intent of the original text, while making it difficult to attribute the authorship to the original author. following past literature @xcite @xcite , we consider an obfuscation method successful if the obfuscated text satisfies the following three requirements: • style concealment analysis of the obfuscated text does not reveal the original author. this is usually measured using an authorship attribution model or a threat model @xcite ). ## jamdec we present jamdec, which obfuscates any text without any prior knowledge of the author. jamdec is composed of three main steps: keyword extraction, over-generation, and filtering, which can be implemented on a sentence, paragraph, or full document level. ## experiments we evaluate two versions of jamdec on two benchmarks in distinct domains: scholarly passages and diary-style entries. for baselines, we consider three state-of-the-art methods for authorship obfuscation: mutant-x @xcite , @xcite , and stylometric @xcite , and a paraphrasing method @xcite . as a stronger baseline, we also consider using zero-shot prompting of gpt3.5 175b which is orders of magnitude larger @xcite . for further details, see appendix g and for access to the code see here. ## conclusion in this work, we introduced jamdec, a novel approach to user-controlled, inference-time authorship obfuscation which utilizes only small, opensource language models. this technique involves three key stages: keyword extraction, constrained diverse beam search, and filtering, offering users fine-grained control over the process and yielding personalized outcomes dependent on the user's needs. we showed experimentation on two diverse datasets, and demonstrated that jamdec outperformed over existing state-of-the-art methods in authorship obfuscation, while also showcasing its competitive performance against significantly larger models like gpt3.5. our findings underscore the promise of jamdec as an effective strategy for authorship obfuscation, harnessing the capabilities of smaller, openly available models to achieve results on par with their larger counterparts.