Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Model (2021)

Abstract: With advances in neural language models, the focus of linguistic steganography has shifted from edit-based approaches to generation-based ones. While the latter’s payload capacity is impressive, generating genuine-looking texts remains challenging. In this paper, we revisit edit-based linguistic steganography, with the idea that a masked language model offers an off-the-shelf solution. The proposed method eliminates painstaking rule construction and has a high payload capacity for an edit-based model. It is also shown to be more secure against automatic detection than a generation-based method while offering better control of the security/payload capacity trade-off.

## introduction steganography is the practice of concealing a message in some cover data such that an eavesdropper is not even aware of the existence of the secret message @xcite . while images, videos, and audio have been dominant cover media @xcite , natural language is a promising choice, thanks to the omnipresence of text @xcite . formally, the goal of linguistic steganography is to create a steganographic system (stegosystem) with which the sender alice encodes a secret message, usually in the form of a bit sequence, into a text and the receiver bob decodes the message, with the requirement that the text is so natural that even if transmitted in a public channel, it does not arouse the suspicion of the eavesdropper eve. for a stegosystem that creates the text through transformation, we refer to the original text as the cover text and the modified text as the stego text. a stegosystem has two objectives, security and payload capacity. security is the degree of how unsuspicious the stego text is while payload capacity is the size of the secret message relative to the size of the stego text. the two objectives generally exhibit a trade-off relationship @xcite . edit-based approaches used to dominate the research on linguistic steganography. arguably, the most effective approach was synonym substitution @xcite @xcite @xcite , where a bit chunk was assigned to each member of a synonym group, for example, '0' to marry and '1' to wed. the cover text she will marry him was then modified to the stego text she will wed him such that the latter carried the secret bit sequence '1'. this conceptual simplicity was, however, overshadowed by the complexity of linguistic phenomena such as part-of-speech ambiguity, polysemy, and context sensitivity. for this reason, edit-based approaches were characterized by the painstaking construction of synonym substitution rules, which were tightly coupled with acceptability checking mechanisms (see @xcite for a review and their own elaborate method). with all these efforts, edit-based stegosystems suffered from low payload capacity, for example, 2 bits per sentence @xcite . with advances in neural language models (lms), edit-based approaches have been replaced by generation-based ones @xcite @xcite @xcite . in these approaches, bit chunks are directly assigned to the conditional probability distribution over the next word estimated by the lm, yielding impressive payload capacities of 1-5 bits per word @xcite . however, it remains challenging for an lm to generate so genuine-looking texts that they fool both humans and machines @xcite even if they do not encode secret messages. it is also worth noting that generation-based stegosystems do not necessarily cut out the need for cover texts, as @xcite and @xcite alice cover text ## experiments we tested the proposed method with several configurations and compared it with a generation-based method. to assess security, we employed automatic discriminators and human adversaries. ## conclusions in this paper, we demonstrated that the masked language model could revolutionize edit-based linguistic steganography. the proposed method is drastically simpler than existing edit-based methods, has a high payload capacity, and allows easy control of the security/payload capacity trade-off. the masked language model is a general framework adopted by many bert-like models, of which attempts to handle longer texts @xcite are particularly relevant to steganography. tailoring the training procedure to steganography is also an interesting research direction.