Zero-shot Generative Linguistic Steganography (2024)

Abstract: Generative linguistic steganography attempts to hide secret messages into covertext. Previous studies have generally focused on the statistical differences between the covertext and stegotext, however, ill-formed stegotext can readily be identified by humans. In this paper, we propose a novel zero-shot approach based on in-context learning for linguistic steganography to achieve better perceptual and statistical imperceptibility. We also design several new metrics and reproducible language evaluations to measure the imperceptibility of the stegotext. Our experimental results indicate that our method produces 1.926\times more innocent and intelligible stegotext than any other method.

## introduction data transmission is generally secured using encryption algorithms to create an unrecognizable ciphertext for secure data transmission @xcite . nonetheless, messages transmitted as ciphertexts may easily arouse suspicion from attackers, and eavesdropping may place a threat on the messages being sent. once the communication channel is discovered, the attackers could alter the messages in any way, causing significant damage. steganography is the key to ensuring communication privacy by concealing sensitive messages within monitored channels. general steganography involves embedding a secret message into a public multimedia carrier, such as text @xcite , image @xcite @xcite , audio @xcite , and video @xcite to create a stego-carrier that is then transmitted via the public channel. as shown in fig. 1 , the problem of secure and covert communication can be viewed as a message transmission between two parties, alice and bob, along with a malicious party, eve. eve will monitor the ## methodology in this section, we present the overall framework of our approach, consisting of three modules: (a) codec module to (de)compress secret text into (from) bitstreams, (b) embedding module to select candidate words from the distribution of stegotext, (c) in-context stegotext generation module to approximate the joint distribution of covertext. the pipeline of our framework is illustrated in fig. 1 . figure 3 also explains the details of the embedding module and the in-context stegotext generation. ## related work linguistic steganography can be divided into three types @xcite , selection-based @xcite , editbased @xcite and generationbased steganography @xcite . however, the low embedding rate of these methods hinders their practical application, giving rise to generation-based steganography. to encode secret messages into textual data, early generative steganography utilized the markov model to obtain conditional probability distributions @xcite ) for text generation. subsequently, the development of nlp has significantly improved distribution modeling. it was first proposed by @xcite to use a supervised recurrent neural network (rnn) to learn the statistical language model of natural texts with a fixedlength coding dictionary. subsequently, @xcite proposed a generation-based steganography based on rnn and huffman coding. yang et al. also proposed that different neural models make a difference, such as the generative adversarial network @xcite and variational autoencoder @xcite . @xcite and @xcite further presented provably secure algorithms for better statistical imperceptibility. while supervised methods attempt to fit models to the source distribution, the restriction of the source distribution leads to a high risk of generating similar but unintelligible text due to the complicated process of embedding secret messages. to address this limitation, models pre-trained on large-scale datasets are used. @xcite proposed that arithmetic coding be used with a pre-trained gpt-2 model. @xcite further proposed an improved huffman encoding algorithm, which is intended to adjust embedding rates dynamically. @xcite improved the previous works with self-adjusting arithmetic coding for near-imperceptible text. nevertheless, all of these training-free approaches employ a similar paradigm for generating the context's next sentence, making them impractical in actual usage. ## conclusion in this work, we present a zero-shot approach based on in-context learning for linguistic steganography. for the llm to generate a similar sentence, samples of covertext serve as the context for the llm in the qa. we also integrate several techniques to improve the text generation process for better readability and imperceptibility of the stegotext. our experimental results exhibit the high imperceptibility of our approach compared to other methods. in addition, the stegotext is more intelligible and resembles the covertext in many aspects. we hope this work will be a step towards the more practical applications of linguistic steganography.