Perfectly Secure Steganography Using Minimum Entropy Coupling
Authors: Christian Schroeder de Witt
Samuel Sokota
J Zico Kolter
Jakob Nicolaus Foerster
Martin Strohmeier
Venue: ICLR 2023 poster (2023)
Keywords: Information-Theoretic Steganography
Minimum Entropy Coupling

Abstract: Steganography is the practice of encoding secret information into innocuous content in such a manner that an adversarial third party would not realize that there is hidden meaning. While this problem has classically been studied in security literature, recent advances in generative models have led to a shared interest among security and machine learning researchers in developing scalable steganography techniques. In this work, we show that a steganography procedure is perfectly secure under Cachin (1998)'s information theoretic-model of steganography if and only if it is induced by a coupling. Furthermore, we show that, among perfectly secure procedures, a procedure is maximally efficient if and only if it is induced by a minimum entropy coupling. These insights yield what are, to the best of our knowledge, the first steganography algorithms to achieve perfect security guarantees with non-trivial efficiency; additionally, these algorithms are highly scalable. To provide empirical validation, we compare a minimum entropy coupling-based approach to three modern baselines---arithmetic coding, Meteor, and adaptive dynamic grouping---using GPT-2, WaveRNN, and Image Transformer as communication channels. We find that the minimum entropy coupling-based approach achieves superior encoding efficiency, despite its stronger security constraints. In aggregate, these results suggest that it may be natural to view information-theoretic steganography through the lens of minimum entropy coupling.

Published as a conference paper at ICLR 2023
PERFECTLY SECURE STEGANOGRAPHY USING MINIMUM ENTROPY COUPLING
Christian Schroeder de Witt" FLAIR, University of Oxford cs cs@robots. ox. ac. uk
Samuel Sokota"
Carnegic Mellon University
asokota@andrew cmu edu
Zico Kolter Carnegie Mellon University zkolterêcs cmu. edu
Jakob Foerster
FLAIR, University of Oxford j jakoberebot ox. ac.
Martin Strohmeier
armasuisse Science + Technology martin. m.
ch
ABSTRACT
Steganography is the practice of encoding secret information into innocuous con- tent in such a manner that an adversarial third party would not realize that there is hidden meaning. While this problem has classically been studied in security literature, recent advances in generative models have led to a shared interest among security and machine learning researchers in developing scalable steganography techniques. In this work, we show that a steganography procedure is perfectly secure under Cachin [9998's 's information only if is induced by coupling. Furthermore, we show that, among perfectly secure procedures, a procedure is maximally efficient if and only if it induced by a minimum entropy coupling. These insights yield what are, to the best of our knowledge, the first steganography algorithms to achieve perfect security guaran- tees with non non-trivial efficiency; additionally, these algorithms are highly scalable. To provide empirical validation, we compare minimum entropy coupling based approach to three modern baselines arithmetic coding, Meteor, and adaptive dynamic grouping-
eevoretic-mode of steganography if and
GPT-2, WaveRNN. and Image Transformer as commu- nication channels. We find that the minimum entropy coupling-based approach achieves superior encoding efficiency, despite stronger security constraints. aggregate, these results suggest that it may be natural to view information-thew theoretic steganography through the lens minimum entropy coupling.
INTRODUCTION
In steganography (Blum & Hopper 2004 Cachin 2004), the goal, informally speaking, is encode a plaintex! message into another form of content (called stegotext) such that it appears similar enough innocuous content (called covertext) that an adversary would not realize that there is hidden meaning. Because steganographic procedures hide the existence of sensitive communication altogether, they provide a complementary kind of security to that of cryptographic methods, which only hide the contents of the sensitive communication not the fact that it is occurring.
In this work, we consider the information -theoretic model of steganography introduced in (Cachin 1998). In Cnchin (1998)'s model, the exact distribution of covertext is assumed to be known to all parties. Security is defined in terms the KL divergence between the distribution of covertext and the distribution of stegotext. A procedure is said to be perfectly secure if it guarantees a divergence of zero. Perfect security is a very strong notion of security, as it renders detection by statistical or
Equal contribution

---

Published as a conference paper at ICLR 2023
human analysis impossible. To the best of our knowledge, the only existing algorithms that achieve both perfect security and non non-trivial efficiency are limited to specific distributions of covertext, such as the uniform distribution, making their applicability limited.
The main contribution of this work is formalizing a relationship between perfect security and couplings of distributions- that is, joint distributions that marginalize to prespecified marginal distributions. We provide two results characterizing this relationship. First, we show that a steganographic procedure perfectly secure if and only if it is induced by couplings between the distribution of ciphertext (an encoded form of plaintext that can be made to look uniformly random) and the distribution of covertext. Second, we show that, among perfectly secure procedures, a procedure is maximally efficient if and only if it is induced by couplings whose joint entropy are minimal that is, minimal entropy couplings (MECs) (Kovačević et al. 2015).
While minimum entropy coupling is an NP-hard problem, there exist O(N log N) approximation algorithms (Kocaoglu et al. [017]] Cicalese et al. 2019] Rossi 2019) that are suboptimal (in terms of joint entropy) by no more than one bit, while retaining exact marginalization guarantees. Furthermore, Sokota et al. (2022) recently introduced an iterative minimum entropy coupling approach, which we call iMEC, that iteratively applies these approximation procedures to construct couplings between one uniform distribution and one autoregressively specified distribution, both having arbitrarily large supports, while still retaining marginalization guarantees. We show that, because ciphertext can be made to look uniformly random, and any distribution of covertext can be specified autoregressively, iMEC can be leveraged to perform steganography with arbitrary covertext distributions and plaintext messages. Excitingly, to the best of our knowledge, this represents the first instance of steganog- raphy algorithm with non-trivial efficiency and perfect security guarantees that scales arbitrary distributions of covertext.
In our experiments, we evaluate iMEC using GPT-2 (Radford et al. 2019) language model), WaveRNN (Kalchbrenner et al. 2018) (an audio model), and Image Transfomer (an image model) as covertext distributions. We compare against arithmetic coding (Ziegler et al. 2019). Meteor (Kaptchuk et al. 20 and adaptive dynamic grouping (ADG) (Zhang et | J. other recent methods for performing information sherretie-teeganographyy with deep generative models. To examine empirical security, we estimate the KL divergence between the stegotext and the covertext for each method. For iMEC, we find that the KL divergence on the order of numerical precision, in agreement with our theoretical guarantees. In contrast, arithmetic coding, Meteor, and ADG yield KL divergences many orders of magnitude larger, reflecting their weaker security guarantees. To examine encoding efficiency, we measure the number of bits transmitted per step. We find that iMEC generally yields su- perior efficiency results to those of arithmetic coding, Meteor, and ADG, despite its stricter constraints.
We would summarize our theoretical results showing that minimum entropy coupling-base based approaches are the most efficient perfect security approaches and our empirical results as showing that minimum entropy coupling-based approaches can be more efficient than less secure alternatives. In aggregate, we believe that these findings suggest that it may be natural to view information-theon steganography through the perspective of minimum entropy coupling.
BACKGROUND
In the first half of this section, we review the information-theone model of steganography introduced by Cachin 1998). In the second half, we review couplings and minimum entropy couplings (Kovače vić et 2013).
2.1 AN INFORMATION- THEORETIIC MODEL FOR STEGANOGRAPHY
Problem Setting The objects involved in information-theerrtie steganography can be divided into two classes: those which are externally specified and those which require algorithmic specification. Each class contains three objects. The externally specified objects include the distribution over plaintext messages M, the distribution over covertext C, and the random source generator.
The distribution over plaintext messages M may be known by the adversary, but is not known by the sender or the receiver. However, the sender and receiver are aware of the domain M over which
2

---

Published as a conference paper at ICLR 2023
Private Key
K
Key Generator
Receiver
Plaintext Message
M
Stegotext S
Plaintext Generator
Sender
Random R Source
Random Source Generator
Adversary
Figure 1: A graphical depiction of steganography. The sender receives a plaintext message, source of randomness, and private key, and outputs a stegotext. The receiver receives the same private key as the sender, along with the stegotext. The adversary also receives the stegotext.
M ranges. The sampled plaintext message M is explicitly known by the sender, but not to the receiver the adversary.
The covertext distribution C is assumed be known by the sender, the receiver, and the adversary.
The random source generator provides the sender with mechanism take random samples from distributions. This random source is known to the sender but not to the receiver or adversary.
The objects requiring algorithmic specification, which collectively referred to stegosystem, are the key generator, the encoder, and the decoder.
The key generator produces private key K in the form of a binary string. This private key is shared between the sender and receiver over secure channel prior to the start of the stegoprocess and can be used to coordinate encryption and decryption. The key generation process may be known to the adversary, but the realization of the key is not.
The encoder takes private key K, a plaintext message M, and a source of randomness as input and produces stegotext S in the space of covertexts C.
The decoder takes private key K and stegotext S as input and returns an estimated plaintext message M.
Many of the objects described above are depicted visually in Figure
In this work, we are interested in stegosystems that possess perfect security.
Definition 2.1. (Cachin) 1998) Given covertext distribution C and plaintext message space ML, stegosystem is e-secure against passive adversaries if the KI. divergence between the distribution of covertext C and the distribution of stegotext S less than e; ie.. KI,C, < It is perfectly secure if the KL divergence is zero; i.e., KL(C,S) =
In other words, steganographic system is perfectly secure if the distribution of stegotext S commu- nicated by the sender exactly the same as the distribution of covertext C.
In addition to security, it is desirable for stegosystems to transmit information efficiently. Mutual information between messages and stegotexts is one way to quantify efficiency.
Definition 2.2. The mutual information I(M; H(M) H(M S) between the message M and stegotext is the expected amount of uncertainty the message M that is removed by conditioning the stegotext S.
Methodological Outline A common class of stegosystems uses two-step encoding and two-step decoding processes, as described below:

---

Published as a conference paper at ICLR 2023
1. The sender uses the private key K to injectively map the plaintext message M into ciphertext
X = {0, 1 in such a way that the induced distribution over ciphertext X is uniformly random, regardless of the distribution of M
2. The sender uses (potentially stochastic) mapping ʃ: X ~ (C to transform the ciphertext X into stegotext S (which exists in the space of covertexts C).
The receiver estimates the ciphertext X from the stegotext S.
4. The receiver inverts the estimated ciphertext X to a plaintext message M with private key
In the outline above, steps I and 4 can be accomplished using standard operations in steganography literature and, thus, are left implicit in much of the remainder of the paper. Our methodological contribution specifically concerns steps and
2.2 COUPLING
The central tool we will use to approach steganography is based on coupling distributions. Let X' and Y' be probability distributions over finite sets X and W. A coupling 7 of X and Y is joint distribution over X x Y such that, for all x E X, Ey v'r (xx and such that, for all
E Y, En 7(x', y) = y()). In other words, a coupling is a joint distribution over X and Y that marginalizes to X' and y. respectively. Note that, in general, there may be many possible couplings for distributions X' and y/
Minimum Entropy Coupling Let T(X, denote the set of all couplings. minimum entropy coupling (MEC) is an element of T(X, with minimal entropy. In other words, MEC is y E I' y) such that the joint entropy H (7) = Em.y ((z, y) log (xx y) is no larger than that of any
other coupling / E ()
STEGANOGRAPHY AND MINIMUM ENTROPY COUPLING
We are now ready to explain our main results. First, we define what it means for an encoding procedure to be induced by a coupling.
Definition 3.1. We say that an encoding procedure f: X C induced by coupling if there exists E I'X'' C) such that for all x E E (,C = -CC=c - X=x).
We formalize relationship between steganographic encoding procedures with perfect security and steganographic encoding procedures that are induced by couplings.
Theorem 1. A steganographic encoding procedure is perfectly secure if and only if it is induced by coupling.
Proof. (+) Assume that steganographic encoding procedure f: X ~4 C is perfectly secure. Define marginal probability distribution y(=rr) = xxx and a conditional probability distribution (C=c X=x) P()()) Note that, together, (X) and 7C' X) define a joint probability xxtribbutionnxXxxx_,__x C=c) (X=x)r(CC=e X X=r). It suffices to show that this joint distribution a coupling. To see this, first, consider that, by construction, marginalizing over C. yields X. Second, note that, by the definition of perfect security, for all C. we must have ExxxP(((A = C(c). Thus, because P(f(X)=c) = (C'=c X), marginalizing over X must yield C.
(-) Assume that there is some 7 E C( that induces steganographic encoding procedure f: X mm C. Then, by definition, for all E C, we must have Ex~xy(C=c) X) = C(c). Thus, because P(f(X)=c) (C'c X), it follows that is perfectly secure.
Next, we formalize a relationship between maximally efficient perfectly secure steganographic encoding procedures and stegagraphic encoding procedures induced by minimum entropy couplings.
For example, if K drawn from uniform random distribution, bin(M) denotes deterministic binariza- tion of M, and XOR represents the component-wiss exclusive-or function, then X XOR(bin(M), K) guaranteed to be distrihuted uniformly randomly, regardless For the example in footnote the receiver can recover the binarized message bin( (M) using the mapping x XOR(X, X, K) and invert the hinarization recover the plaintext M.
distribution of messages M.
4

---

Published as a conference paper at ICLR 2023
Theorem 2.
Among perfectly secure encoding procedures. procedure f: X ~s C maximizes the mutual information ((; if and only if f is induced by a minimum entropy coupling.
Proof. First, note that, because the mapping from plaintext messages is injective, have I(M I(X;5) 5). Thus, it suffices to prove the analogous statement for :(::S) Also, note that, because the theorem's 's statement is specific to perfectly secure f. we know from Theorem that we need only concern ourselves with procedures induced by E C))
(+) Assume that y maximizes I(X; S)). Then, because I(X; H(X) H(S) H S), ? must also maximize the right-hand side of the equality. Furthermore, because K(X) is fixed and H(S) is equal to M((C) by definition of perfect security, follows that y must minimize H(X,
Then, by definition, is a minimum entropy coupling.
(=) Assume that y minimum entropy coupling. Then, because H(X; H(X) H(S) I(X, S), y must also minimize the right-hand side of the equality. Furthermore, because XX fixed and HSS) is equal to H(C) by definition of perfect security, it follows that y must maxi- mize S).
Informally, Theorem I can be viewed as stating that the class of perfectly secure stenographic procedures and the class of set of couplings are equivalent; Theorem can be viewed as stating that the subclass of perfectly secure procedures that maximize mutual information and the set of minimum entropy couplings are equivalent. On one hand, these theorems illuminate an avenue toward achieving efficient and perfectly secure steganography. On the other hand, they show that any steganographic procedures that achieve perfect security and that any steganographic procedures that achieve perfect security with maximal efficiency do so only by virtue of the fact that they are induced by couplings and minimum entropy couplings, respectively.
4 A MINIMUM ENTROPY COUPLING BASED APPROACH TO STEGANOGRAPHY
To turn the insights from the previous section into an actionable steganography algorithm, we require MEC techniques. While computing an exact MEC an NP-hard problem, Cicalese et al. (2019) and Rossi (2019) recently showed that it is possible to approximate MECs in N log N time with a solution guaranteed to be suboptimal by no more than one bit. These algorithms are fast enough to compute low entropy couplings for distributions with small supports. Even more recently, (Sokota et al. (2022) introduced an iterative minimum entropy coupling (iMEC) approach that uses an approximate MEC algorithm as subroutine to iteratively couple distributions with arbitrarily large supports, so long as one of the distributions is uniform and the other can be specified autoregressively. This approach provably produces couplings, meaning that exact marginalizaaton (and therefore perfect security) is guaranteed. We describe how iMEC can be directly applied as a steganography algorithm below.
Croblem,,Cicalse
Iterative Minimum Entropy Coupling Assume that X' is uniform distribution and let X, x... Xn = X and C₁ X ... x Cm C be factorizations over the spaces that X' and C range. iMEC implicitly defines a coupling 7 between X' and C using procedures that iteratively call an approximate MEC algorithm as a subroutine. These procedures can sample S ~ 7C' X=x) and compute y(X C=s) for a given x and s, respectively. We will call these operations encoding and decoding.
Algorithm 1 Sampling S ~ |
procedure ENCODE(Z, X,C)
Initialize uniform distribution 14 over X, for each é = 1,..., n.
for j 1...., do 1 + argmax, H (w:)
>. Select maximal entropy block
2i MEC( C(C, (Cgg-1-Sg--1)
Call approximate MEC subroutine
S, ~ (C,C, Xx=1T-)
& Sample encoded stegotoken
p4- + (X,, |C,-S)
Update ciphertext posterior
end for return S end procedure
Return stegotext
5

---

Published as a conference paper at ICLR 2023
Algorithm 2 Computing 7X C=s)
procedure DECODE(S, X.
Initialize uniform distribution 14, X, for each i 1, ,,..., n. for j = 1..... m do i* argmax, (1:)
Select maximal entropy block
7j < MEC( CCCC C15-1=819-1))
D Call approximate MEC subroutine
p4- 7x(x; (C==s))
Update ciphertext posterior
end for
return x + II, (x;)) end procedure
b Return ciphertext posterior
Encoding, described in Algorithm I and decoding, described in Algorithm share similar structures. Encoding greedily encodes information about into S, using the induced posterior over to maximize coupling efficiency; decoding simply reproduces this posterior and selects the maximum posterioni.
EXPERIMENTS
To validate iMEC as a steganography algorithm. we evaluate it empirically against arithmetic coding (Ziegler et al. 2019), Meteor (Kaptchuk et al., (021), and ADG (Zhang et al. (202T) on four different covertext types. We also include variant of Meteor that employs bin-sorted probabilities that we call Meter:reorder.
5.1 EXPERIMENT SETUP
Our first covertext distribution consists of uniformly random noise of dimension 40 and mean covertext entropy of Hc 5.32 bits per token. Our other channels are designed to mimic the natural distributions of three widely utilized modes of human communication: text, speech, and images. The second and third covertext distributions are variants of GPT-2 (Radford et al. 2019) with 12 attention modules (Wolf et al. 2020) conditioned on 1024-character strings from the Wikitext- 103 dataset (Merity et al., 2016). The second performs top-k sampling from a normmallleed categorical distribution over the 40 highest-probability outputs. The third performs nucleus sumpling (Holtzman al. 2020) from the highest-probability outputs that together comprise 90% of the raw channel entropy (Radford et al.) 2019). The fourth covertext distribution consists of text-to- so-eech (TTS) pipeline (Yang et al. (022) based on Tacotron-2 (Shen et al., 2018) mel-spectogram encodings, followed by WaveRNN generative model (Kalchbrenner et al. 2018) pretrained the LJSpeech dataset (Ito & Johnson 2017) with a mean covertext entropy of Hie = 0.35 bits per token. We condition the WaveRNN model using text generated by GPT-2 that we convert into audio data using mel- mel-spectogram encodings. The fifth covertext distribution is over an unconditional transformer- based generative model (Parmar al. 2018 of CIFAR-JO (Krizhevsky al. 2009) RGB images of dimensions 32 x 32.
Our implementation of iMEC makes use of the approximative minimum entropy coupling procedure described in Algorithm 1 of (Kocaoglu et al.) 2017) as its subprocedure. We show results for three different iMEC block sizes (10 bits, 16 bits, and 20 bits). The number of bits measures the initial entropy of particular H(1,). The larger the block size, the more burden iMEC places on the approximate MEC subroutine-thi may result in lower entropy couplings but have higher computational cost. We used the original implementations of arithmetic coding (Ziegler et al. 2019). Metcor/Meteor.cordder (Kaptchuk et al. 2021). and ADG (Zhang et al. 2021 for the basclines.
All experiments performed
AMD Ryzen Threadripper PRO 3955WX with 16 physical cores and 2x NVIDIA GeForce RTX 3090 GPUs. Apart from model queries, iMEC encoding and decoding occupies just single CPU core, while arithmetic coding, ADG, and Meteor make of multiple CPUs, and Meteor.reorder also makes use of GPUs during encoding and decoding.
All ciphertexts 80-bit bitstrings sampled uniformly random. We measure the encoding efficiency of each method by measuring the amount of covertext required to transmit these bitstrings. We tuned the
of each method yield the best performance on this task. For iMEC, stop

---

Published as a conference paper at ICLR 2023
10
107
KL(SSeegtexx]Cam Moloc//ovveeext
10:
IMEC[20 bits]
10a
IMEG]16 bits)
IMEC
float32
10-
ADG
Mateor
floa 4
Uniform-40 GPT-2 [topk 40) GPT-2 (90%]
WaveRNN imageTransfermer
Figure 2: Kullback-Leibler divergences between the stegotext distribution and covertext distribution for each method.
Encoding Efficiency
0.8
3.6
IMEC129
INEC[10 bits]
2.4
Arthmetic
9.2
Neteerrtenrder
Uniform-40
GPT-2 [top40] GPT-2 (90%]
WaveRNN Imagelranstormer
Figure 3: Comparative encoding efficiencies of iMEC with baselines. Each method is evaluated at tuned hyperparameters.
transmission when the entropy of every posterior block is less than 0.1, i.e., Vi, H(a)) < 0.1. We found the error rate for this threshold be lower than in 10 bits (see Figure 5). For arithmetic coding and Meteor/Metteorreordee we performed a grid search over precision values of 8, 16, 32 and 48 bits. We find that both arithmetic coding and Meteor get stuck a seemingly infinite loop about 3% of the time (see Figure For simplicity, simply discard these runs from investigation; this may result in overestimation of these algorithms' encoding efficiencies. Errorbars indicate estimates of 95% confidence intervals computed from the central limit theorem over 1000 runs.
5.2 RESULTS
Covertext Divergence We first investigate the empirical security of each method. We follow the precedent of previous work (Ziegler al.) 2019: Zhang et al. 2021) and show KL(S, C), which can easily be estimated from samples. We present the results in Figure 2 We used float64s for the experiments; thus, we expect the divergence of methods with perfect security should generally be on similar order of magnitude as float64s. Overall, our results suggest that that is true true-iMEC's KI. divergence upper bounded by numerical precision in every setting. On the other hand, arithmetic coding and ADG possess divergences that consistently many orders of magnitude above the precision of the data type. While Meteor and Meteor.reorder exhibit relatively better performance on uniform covertext distributions, they incur larger divergences for the complex distributions (including a divergent KL divergence for Meteor:reorder on WaveRNN).
Encoding Efficiency We next compare encoding efficiencies and bit rates. We say bit rate mean the number of bits encoded per token. We say encoding efficiency to mean the bit rate normalized to the mean covertext entropy per token. We show the results for encoding efficiencies in Figure 3 the
7

---

Published as a conference paper at ICLR 2023
IMEC[20 bits)
[EC[16 biss
10T4
IMEC[10
Arthiettcc
ADG
GPT3
WaveRRNN
10-3
Uniform40 GPT-2 (topk 40) GPT-2 (90%] WaveRNN imageTranstermer
Figure 4: Comparative speed evaluation of iMEC and baselines. Horizontal lines indicate the amount of time required for model inference for GPT-2, WaveRNN, and Image Transformer.
results for bit rate are shown in Figure 7 in the appendix. Surprisingly, despite its perfect security constraints, iMEC with block size 20 bits yields superior encoding efficiency the baselines across all three of our realistic distributions. We also observe that iMEC with block size of 16 bits generally yields competitive or superior performance the other methods, and that 10 bit block sizes also performs competitively. Impressively, at a standard encoding frequency of 24kHz, iMEC's performance for WaveRNN would allow to encode a private message of 225 kilobytes just 30 seconds of TTS voicemail--ufficee
for sending compressed images.
Performance among the baselines varies. Arithmetic coding achieves greater efficiency than iMEC[10 bits] and 1MEC[16 bits] on the uniform covertext distribution and comparable performance with iMEC[10 bits] on GPT-2 90%. ADG achieves the best efficiency of all methods on the uniform covertext distribution, but is the least efficient otherwise, especially WaveRNN. Meteor is generally less efficient than all iMEC variants and arithmetic coding. Meteor:reorder achieves greater efficiency than 1MEC[10 bits] for GPT-2 90%, but is otherwise less efficient than all iMEC variants.
Speed Lastly, we examine the speed of each algorithm. We show results in Figure While, in the previous section, observed that increasing iMEC's block size can improve encoding efficiency, we see here that this improved efficiency does not come without cost. While 10 bit blocks require an order of magnitude less time than model inference, 16 bit blocks require the same order of magnitude of time as model inference, and 20 bit blocks require an order of magnitude more time than that, The wall-clock time of arithmetic coding and Meteor are generally comparable to that of the 10 bit blocks, while the wall-clock of Meteor.reorder and ADG vary somewhat significantly depending on the task. We believe entropy coupling will allow some of the cost of coupling to be distributed across multiple cores, making the block sizes we experiment with here much cheaper.
possible (perhaps even likely) that innovations in approximate minimum
6 RELATED WORK
Provable Perfect Security One body of related work concerns algorithms with provably perfect security. One case in which perfect security is possible is when the covertext distribution is uniform (Cachin 1998). In this case, perfect security can be achieved by embedding the message in uniform random ciphertext over the same domain as the covertext. However, constructing algorithms that both guarantee perfect security and transmit information covertext distributions has proved challenging. One notable result is that of Wang & Moulin (2008)
non-vanishing rates for more general
who show that, in the case that tokens of the covertext are independently identically dustributed, public watermarking codes (Moulin & O' D'Sllliaan] 2003) that preserve first order statistics can be used to construct perfectly secure steganography protocols of the same error rate. Another relevant line of research is that Ryabko & Ryabko (2009), who show that perfectly secure steganography can be achieved in the case that letters of the covertext independently identically distributed under the weaker assumption of black box access to the covertext distribution. follow up work, Ryabko & Ryabko| (20IT) generalize their earlier work to a setting in which the letters of the covertext need only follow k-order Markov distribution. Unlike
& Merhay 2003 2004)
8

---

Published as a conference paper at ICLR 2023
these works, our approach does not make any assumptions on the structure of the distribution of covertext, though, unlike Ryabko & Ryabko (201T) we do assume that this distribution is known.
iiformmttionTheorreee Steganography with Deep Generative Models Another body of related work concerns scaling information- thhooreticccccccccoo steganography to covertext distributions represented by deep generative models. Most work in this literature leverages the relationship between compression and steganography (Sallee) (003), wherein the steganography problem is viewed as a compression problem for the covertext distribution. Existing algorithms this class are based on Huffman coding DDai & Cai 2019) and arithmetic coding (Zieglrr et al., 2019 Kaptchuk et al. 202T Chen et al. 2022). While some of these works possess e-security guarantees, they are not perfectly secure unless the covertext distribution is compressed exactly into the ciphertext distribution, which is not generally the case. Outside of coding-hasse
methods, Zhang et al. (202T) describe another method that they call ADG, which, similarly to coding based approaches, possesses nearly perfect-but not perfect-securicc guarantees. Our experiments confirm the lack of perfect security guarantees for these methods-inndect we observed that arithmetic coding (Ziegler et al., 2019). Meteor (Kaptchuk) (et al.) (022),, and ADG (Zhag et al. 202T) incurred KL divergences many orders of magnitude greater than the level of numerical precision we used.
7 CONCLUSION AND FUTURE WORK
In this work, we showed 1) that perfect steganographic security is equivalent to a coupling problem and 2) that achieving maximal transmission efficiency among perfect security procedures is equivalent to a minimum entropy coupling problem. These findings may suggest that coommation-theoretee steganography is viewed most naturally through the lens of minimum entropy coupling. Furthermore, using recent innovations in approximate and iterative minimum entropy coupling (Kocaoglu lu et al., 2017: Sokota et al. 2022). we showed that this insight is also practical, leading to a MEC-based approach that is scalable to covertext distributions expressed by deep generative models. In empirical evaluations, we found that our MEC-based approach's 's divergence from the covertext distribution is on the order of numerical precision, in agreement with our theoretical results. Moreover, we observed that it can achieve superior encoding efficiency compared to arithmetic coding- coding-based methods. The existence of a scalable stegosystem with perfect security guarantees and high empirical efficiency may suggest that statistical steganalysis carries little value within Cachin (1998)'s 's model.
The most significant limitations of our work arise from the assumptions of Cachin (9988)s 's cnormatiio-thhoreeti formulation of steganography. First, [aahhnn ( (1998) assumes that the ad- versary is passive and that the stegotext arrives to the receiver unperturbed. While this assumption is standard among related work and may hold for many digital transmission channels, it is unrealistic in other settings. One direction for future work is to extend variant of the minimum entropy coupling perspective of steganography to settings in which the channel medium is noisy. This may be possible by taking inspiration from ideas in error correction literature. Second, Cachin (1998) assumes that white box access to the covertext distribution is available. Unfortunately, this assumption often unrealistic-an even modern deep generative models struggle accurately approximate complex distributions (though it is expected that this issue will be somewhat ameliorate over time). Furthermore, realistic scenarios, the distribution of "normally" occurring content may shift over time and depend on other external context, making it difficult to capture (though it would also affect the adversary's ability to detect distributional anomalies). While there exists information- -hhooretcc steganography literature that does not assume white box access to the covertext distribution under the name universal steganography, it is difficult to achieve general purpose guarantees. Thus, dropping this second assumption may be more challenging, suggesting that minimum entropy coupling perspective on steganography may be best suited to settings which the covertext distribution can be modeled accurately.
8 REPRODUCIBILITY
An open source implementation of iMEC is available at sch scchreedrrdderitt/p/rffcltt---ss otgprr/eeeollyoseoreeeeeeeenggnn
https://github.com

---

Published as a conference paper at ICLR 2023
9 ACKNOWLEDGMENTS
We thank Philip Torr, Yiding Jiang, Pratyush Maini, and Jeremy Cohen for helpful feedback. This research was supported by the Cooperative AI Foundation, Armasuisse gxeenceTTechologe and the Bosch Center for Artificial Intelligence. CSDW has additionally been supported by an EPSRC IAA Doctoral Impact Fund award kindly hosted by Prof. Philip Torr.
REFERENCES
Manuel Blum and Nicholas Hopper. Toward theory of steganography. 2004.
Christian Cachin. An infommation-thooelee model for steganography. In David Aucsmith (ed.), Information Hiding, pp. 306-318, Berlin, Heidelberg, 1998. Springer Berlin Heidelberg. ISBN 978-3-540-4938-8.
Christian Cachin. Digital steganography, 2004.
Kejiang Chen, Hang Zhou, Hanqing Zhao, Dongdong Chen, Weiming Zhang, and Nenghai Yu. Dsstibution-preseriing steganography based on text-to-speech generative models. IEEE Transac- fions Dependable and Secure Computing, 19(5):3343 333233333366,,,3- 2022. doi: 10.1 10.1109/TDSC.2221. 3095072.
Ferdinando Cicalese, Luisa Gargano, and Ugo Vaccaro. Minimum-entropp couplings and their applications. IEEE Transactions on Information Theory, 65:3436-3451, 2019.
Falcon Dai and Zheng Cai. Towards near- inaerrrppeee steganographic text. In Proceedings of the 57th Anmual Meeting of the Association for Computational Linguistics. pp. -300, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.18533/11/11914422 URL
httts:://ccaattai anthhcllnnttoooooooo
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The Curious Case of Neural Text Degeneration. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL /ttt:ptttt////peeeeee/
met//ors idd rygGQyrFvH
Keith Ito and Linda Johnson. 2017.
The lj speech dataset.
httpp://heithito..comm
LJJ
Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lock- hart, Florian Stimberg, Aaron Oord, Sander Dieleman, and Koray Kavukeuoglu. Efficient Neu- ral Audio Synthesis. In Proceedings of the 35th International Conference on Machine Learn-
ing. pp. 2410-2419. PMLR, July 2018, URL https: httL:tttts/://peeddiiiee http:///poeeeeggs.m....c mwr. www.sppress/eee / kalchbrennerl 8a. html
ISSN: 2640-3498.
Gabriel Kaptchuk, Tushar Jois, Matthew Green, and Aviel D. Rubin. Meteor: Cryptographically se- cure steganography for realistic distributions. In Proceedings of the 2021 ACM SIGSAC Conference
on Computer and Communications Security, CCS '21, pp. 1529-1548. New York, NY, USA, 2021. Association for Computing Machinery. ISBN 978 9781450384544 doi: 10. 011145/3460110.44455555
URL http::ttt::/// //tl: http///llog///////////oo///////a www.o0..00000..... 1145/ /015204060022 3484550
M. Kocaoglu, A. Dimakis, S. Vishwanath, and B. Hassibi. Entropic Causal Inference. In AAAI, 2017.
Mladen Kovačević, Ivan Stanojević, and Vojin Šenk. On the entropy of couplings. Informa- fion and Computation, 242:369 382, 2015. ISSN 0890-5401. doi: http://doii.rr//0.1016
ic.2015.04.003. URL http:: wwwp///wwww//o//////////////wo www.ssiience.. rect. com wtam/sciencee wwc.eenne//aiiile/p.ii.. S08 00890540115000455
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer Sentinel Mixture Mod- els. CoRR, abb//160.007843, 2016. /tt.a//tt///axavoooo//////////// 07843 arXiv:
1609.07843.
10

---

Published as a conference paper at ICLR 2023
P. Moulin and J.A. O'Sullivan. Information-theoreet analysis of information hiding. JEEE Transac- tions on Information Theory, 49(3):563- 593, 2003. doi: 10. 10/11000.200...00000 908134.
Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and
Dustin Tran. Image Transformer. In Proceedings of the 35th International Conference on Machine Learning, pp. 4055-4064. PMLR, July 2018. URL
mlr press/
v80/pa http/parmar11.. html]
ISSN: 2640-3498.
Alec Radford, Jeff Wu,
Rewon Child, D. Luan, Dario Models are Unsupervised Multitask Learners
Amodei, and Ilya wn-
Sutskever. Language
defined, 2019.
URL https: www. www.eetiiccoooa..rroooooooo http:s//www.eeeeeccccccooooooao.
Larguage
Radford-Wu/ -aaearnnrr--daod-dar
9405cc0d6 3at.a/ac461199811177777767711333 4dfe
Massimiliano Rossi. Greedy additive approximation algorithms for minimum-entropp coupling problem. In 2019 IEEE International Symposium Information Theory (ISIT), pp. 1127-1131, 2019. doi: 00.1109/SST.2201988444999
Boris Ryabko and Daniil Ryabko. Asymptotically optimal perfect steganographic systems. Problems of Information Transmission, 45: 184-909, 06 2009. doi: 10.1 133/50005294000000099944
Boris Ryabko and Daniil Ryabko. Constructing perfect steganographic systems. Information
and Computation, (0999)) 2011. ISSN 0890-5401. doi: https//ddi..rg//0..001//
URL htt.://sewwwwecceennccreeeecooo htt.://cceeccc.coooo//e/eeeeeerr
www.
50890540111001001
Phil Sallee. Model- steganography, volume 2939, pp. 154-167, 10 2003. ISBN 978-3-540- 21061-0. doi: 10. 00.1007/9778-3350-2422222 4462444--444
Jonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang. Rj Skerrv-Ryan, Rif A. Saurous, Yannis Agiomvrgiannakis, and Yonghui Wu. Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions. In 20/8 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4779 4783, April 2018. doi: 10.1109/ICASSP22011
368. ISSN: 2379-190X.
Samuel Sokota, Christian Schroeder de Witt, Maximilian Igl. Luisa M Zintgraf, Philip Torr, Martin Strohmeier, Zico Kolter, Shimon Whiteson, and Jakob Nicolaus Foerster. Communicating via markov decision processes. In Proceedings of the 39th International Conference on Machine Learning, ICML''22. JMLR.org, 2022.
A. Somekh-Baruch and N. Merhav. On the error exponent and capacity games of private watermarking systems. IEFE Transactions on Information Theory, 49(3):337-562, 2003. doi: 10.1 109/TTT:2002. 808132.
A. Boruchh and Merhav. On the capacity game of public watermarking systems. IEEE Transactions Information Theory, 5003):51--5 2004. doi: 10. 10.1109/TT.220448844990
Ying Wang and Pierre Moulin. Perfectly secure steganography: Capacity, error exponents, and code constructions. IEEE Transactions on Information Theory, 54(6))2706-277 2008. doi: 10.1 10.1100//T..200.921166
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. HuggingFace's 's Transformers: State-of- the-art Natural Language Processing. Technical Report arXiv: 1910.0377, arXiv, July 2020. URL http::/arxiis wr..orggg.. 1 arXiv:1910.03771
[cs] type: article.
Yao-Yuan Yang, Moto Hira, Zhaoheng Ni, Anjali Chourdia, Artyom Astafurov, Caroline Chen, Ching-Feng Yeh, Christian Puhrsch, David Pollack, Dmitriy Genzel, Donny Greenberg, Ed- ward Z. Yang, Jason Lian, Jay Mahadeokar, Jeff Hwang, Ji Chen, Peter Goldsborough. Prabhat Roy, Sean Narenthiran, Shinji Watanabe, Soumith Chintala, Vincent Quenneville- Bélair, and Yangyang Shi. TorchAudio: Building Blocks for Audio and Speech Processing. Technical Report
11

---

Published as a conference paper at ICLR 2023
arXiv:2110.15018, arXiv, February 2022. URL http://awii...rrraa/221...000 arXiv:2110.15018 [cs, eess] type: article.
Siyu Zhang, Zhongliang Yang, Jinshuai Yang, and Yongfeng Huang. Provably secure genera- tive linguistic steganography. In Findings of the Association for Computatiooal Linguistics: ACL-IJCNLP 2021, pp. 3046-3055, Online, August 2021. Association for Computational Linguis- tics. doi: 10. 10..1653/v11200000 Lindiiin ac1.268. URL StLLapeee////llaatttooooooooo//. Findings-ac1.255
Zachary Ziegler, Yuntian Deng, and Alexander Rush. Neural linguistic steganography. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-IJCNLA) pp. 1210- 1215. Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.
URL URL https: / /acl antho 10/00///1110/1//11111115
12

---

Published as a conference paper at ICLR 2023
A APPENDIX
A.1
IMEC ERROR RATE
IMEC UNIF block size: 10
Bit error rate
0.08
IMEC UNIF block size: 20
0.06
0.04
0.02
0.00
10-1
10a
IMEC GPT2 (topk 40) block size: 10
Bit error ratt
0.08
GPT2 (topk 40)
0.06
0.02
0.00
10-1
10"
Belief entropy threshold
Figure 5: Bit error rate as function of threshold size. The error bars shown are the standard deviation of the mean over 100 trajectories.
We show error rate as a function of the belief entropy threshold in Figure As is suggested by the figure, the error rate can be made arbitrarily small by selecting sufficiently small treshold value.
A.2 NON-TERMINATION FREQUENCY FOR ARITHMETIC CODING
Bit error rate [GPT-2] Bit error rate [Uniform]
0.5
Non-Termination Freq Non- inatiooo Freq
0.3
0.4
0.2
0.1
0.0
16
32
48
Arithmetic [topk 40]
0.3
Arithmetic (90%]
0.2
0.2
Precision [bits]
Figure 6: Bit error rate (X) and non-termination frequencies (circles) for arithmetic coding vs. precision hyperparameter. We show estimates over 100 trajectories.
plot of the error rate and non-termination frequency is shown in Figure 6 While we did not observe emors, non-termination occurred with non-negligible probability.
13

---

Published as a conference paper at ICLR 2023
A.3 ENCODING BIT RATE
(90%)
Imagelransfrree
DFrate
GPT2 ptopk 40)
IMEC[20 bit]]
INEC116
INEC[1D
Arthmetic
ADG
Meteer:necrderr
WeveRNN
Uniform 40
GPT-2 (top40)
GPT-2 [90%]
WaveRNN IrageTransformer
Figure 7: The encoding bit rate for each method for each covertext distribution. Horizontal lines the right plot correspond to the mean covertext entropy per token.
An unnormalized version of Figure is shown in Figure 7
A.4 STEGOTEXT SAMPLES
To illustrate the effect bias, we reproduce a sample stegotext for both iMEC (block size 10), as well as Meteorrreorder (precision 32) and the private message length is 20 bytes. Both examples have been mildly post-processed for readability, such as removing special characters and whitespaces.
Context:
Heck horses are dun or grullo (a dun variant) in color, with no white markings. The breed has primitive markings, including a dorsal stripe and horizontal striping on the legs. Heck horses generally stand between 12.2 and 13.2 hands ( (0 and 54 inches, 127 and 137 em) tall. The head large, the withers low, and the legs and hindquarters
iMEC produces the following stegotext:
are short. The neck is wide and thick, characteristic that can be inherited from the male. The face be seen broad head, with pointed toes. The head and neck are often used tool for hunting, though their appearance often depends on their social organization. The legs are
Meteor:reorder produces the following stegotext:
have narrow and angular shape. The fore and hind legs are longer than the head. The tail is broad and short in a shape similar to the neck or neckbone. The front legs have a sharp protrusion that leads from the head to the head but not from the tail. The hind legs have long pangs (2) and lower
Note that Meteor:reorder's high bias seemingly lowers the content quality of the output text.
14


Reviews:
[{"review_id": "nU2atU9dZo", "reviewer": "Reviewer_yanH", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents theoretical proofs of perfectly secure (zero KL divergence) steganography, particularly that it depends on being induced by a coupling (necessary and sufficient condition) and that for efficiency it will also depend on a minimum entropy coupling (that benefit from very recent advances in the approximation of minimum entropy coupling iteratively). The paper presents experiments showing practicality of the assumptions in a (unrealistic) noise-free channel. ", "summary_of_the_review": "The paper is a theoretical paper with thorough derivations and very clear discussions. The result impact is somehow limited to the assumptions of a noise-free channel, but still important.", "strength_and_weaknesses": "\nStrengths:\n1) The paper provides thorough theoretical derivations for this increasingly interesting and important topic\n2) The paper is very clear and methodologically correct\n3) The fact that a theoretical constraint is also efficient is a very interesting result of this paper\n\nWeaknesses:\n1) The fact that the paper assumed the Cachin (1998) model with transmission being made by a noise-free (perfect) channel is, as pointed out by the authors, a drawback that limits the importance and applicability of these results. \n2) Nowadays, important applications of steganography are not limited to 1D signals and thus application on 2D images are of extreme importance, opening relevant applications. I suggest the authors briefly discuss this in the paper, particularly the applicability of the paper results to images.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "d_scjW37OxS", "reviewer": "Reviewer_3EY8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes to view the task of steganography as a minimum entropy coupling problem. The authors show that (1) coupling algorithms yield steganography procedures with perfect security; (2) the approach induced by minimum entropy coupling maximizes information throughput; (3) apply the gained insights to steganography with deep generative model covertext distributions. ", "summary_of_the_review": "I believe that this work holds some novelty for a subgroup in the ML community. While the assumptions made in this work heavily rely on the assumptions made by Cachin might not hold in practical scenarios, the authors provide a “secure” steganography approach from a theoretical standpoint. The authors also showed that their approach is effective in some practical scenarios. ", "strength_and_weaknesses": "## Strengths\n(+) To the best of my knowledge, this is the first work approaching the task of steganography as a minimum entropy coupling problem.\n(+) To the best of my knowledge, this is the first steganography algorithm to achieve perfect security guarantees (in the context of Cachin (1998)’s information theoretic model of steganography)  \n(+) The presented results based on the Kullback-Leibler divergences are convincing (close to zero)   \n(+) The approach is well motivated by the existing theory.  \n\n## Weaknesses\n(-) This work relies on the assumption that “the exact distribution of covertext is assumed to be known”. However, in practice, this might not be the case, since the model of language might not be perfectly known. Hence, for most practical scenarios this approach might not provide perfectly secure steganography.  \n(-) While the results in Figure 2 are close to zero, they are not exactly zero. Does this mean that perfect security is not obtained as claimed?  \n(-) The authors evaluated the security of their approach empirically, however, it is not clear if this method can withstand modern steganalysis methods [A, B, C, D], or human evaluation.   \n\n[A] Modern Text Hiding, Text Steganalysis, and Applications: A Comparative Analysis; Entropy. 2019  \n[B] Linguistic Steganalysis With Graph Neural Networks; IEEE Signal Processing Letters. 2021  \n[C] A fast and efficient text steganalysis method; IEEE Signal Processing Letters. 2019  \n[D] Exploiting language model for efficient linguistic steganalysis; ICASSP 2022  \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "YKJQiKwoJs", "reviewer": "Reviewer_tJp6", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors show that a steganography procedure is perfectly secure under Cachin (1998)’s information theoretic-model of steganography if and only if it is induced by a coupling. Moreover, the authors show that, among perfectly secure procedures, a procedure is maximally efficient if and only if it is induced by a minimum entropy coupling.", "summary_of_the_review": "1) The title of the paper is not appropriate as it is misleading and should be changed. In security field, there is no such thing as \"perfectly secure\" mechanism. Every security measures has its own Pros and Cons depending on circumstances. The title should be revised.\n2) In section 1, it is mentioned, \"A procedure is said to be perfectly secure if it guarantees a divergence of zero.\" This is only true in theoretical sense and not in practical. So, this should be made clear in the statement with appropriate citations.\n3) The motivation in the section 1 is not strong enough. Provide a motivational case study to engage the readers and to prove the importance of such research.\n4) Section 2 (Background) is a bit misleading in written format as it mostly consist of problem formulation rather than background research. Please rename the title and provide background research on other existing related methods.\n5) In section 3 & 4, please define each variables used/mentioned for readability proposes. Not everyone who would be reading this paper would be a field expert in computer security.\n6) Section 5 (experimental evaluation) is weak. Comparative study is lacking and should be included to show the efficacy of the claims being made. Pleas provide that.\n7) In a conference paper, it might be better to segregate future works into a separate section along with discussions. And also have a separate conclusion section.\n8) The paper seems a bit out of scope for this conference and might be better tuned towards computer security related conferences.", "strength_and_weaknesses": "Strength::\nSecurity is an important topic in computing and should be pursued.\n\nWeakness::\nThe paper can be improved based on the following points.\n1) The title of the paper is not appropriate as it is misleading and should be changed. In security field, there is no such thing as \"perfectly secure\" mechanism. Every security measures has its own Pros and Cons depending on circumstances. The title should be revised.\n2) In section 1, it is mentioned, \"A procedure is said to be perfectly secure if it guarantees a divergence of zero.\" This is only true in theoretical sense and not in practical. So, this should be made clear in the statement with appropriate citations.\n3) The motivation in the section 1 is not strong enough. Provide a motivational case study to engage the readers and to prove the importance of such research.\n4) Section 2 (Background) is a bit misleading in written format as it mostly consist of problem formulation rather than background research. Please rename the title and provide background research on other existing related methods.\n5) In section 3 & 4, please define each variables used/mentioned for readability proposes. Not everyone who would be reading this paper would be a field expert in computer security.\n6) Section 5 (experimental evaluation) is weak. Comparative study is lacking and should be included to show the efficacy of the claims being made. Pleas provide that.\n7) In a conference paper, it might be better to segregate future works into a separate section along with discussions. And also have a separate conclusion section.\n8) The paper seems a bit out of scope for this conference and might be better tuned towards computer security related conferences.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "Db-QUO5VDB", "reviewer": "Reviewer_x2x4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors formalize the relationship between perfect security and couplings of distributions, and propose the first instance of a steganography method that has the benefits of non-trivial efficiency and perfect security guarantees. Experiments with GPT-2 and WaveRNN demonstrate the effectiveness of the proposed method. Also, the code is provided for reproducibility. ", "summary_of_the_review": "Steganography is not my primary research area. It seems to me that the paper has a solid theoretical support. ", "strength_and_weaknesses": "The paper is well-written. The authors start by formulizing and explaining the relationship between perfect security and couplings of distributions through two results. Then, the authors accordingly introduce the first instance of steganography algorithm with non-trivial efficiency and perfect security guarantees.\n\nSince steganography is not my primary research area, the theoretical part of this paper is beyond my knowledge. So, I cannot comment its theoretical contribution reaches the standard of ICLR. But it seems to me that the proposed method is well-motivated, based on two findings that the perfect steganographic security is, in fact, equivalent to a coupling problem and achieving maximal transmission efficiency among perfect security procedures is equivalent to a minimum entropy coupling problem. So, I rate this paper as borderline accept for now and will refer to the comments of other reviewers for my final rating.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster