Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a novel neural image steganography method. The fundamental idea is that the sensitivity of neural networks to adversarial perturbations can be leveraged to encode an arbitrary hidden message into imperceptible perturbations. The authors' method heavily outperforms current classical and neural image steganography methods. They also perform several relevant experiments, such as robustness of the method to lossy compression and avoiding steganalysis.

The authors also propose to apply their method to protect persons' identities in pictures by detecting all faces present in a picture, in-painting them with a GAN and hiding the picture of the original face in the GAN-generated image.

main_review: ## Strengths
The observation that neural networks are sensitive to adversarial perturbations can be regarded as a feature and not a bug is very nice.
The method is simple and quite general. The experiments show that the run-time of the algorithm, while appropriate for high-throughput applications,
is reasonable (on the magnitude of a few minutes).
The experiments are relevant and the methodology seemed reasonable with a few minor issues.

The proposed application of hiding images is also very interesting. In my view, its usefulness depends on how easily the optimization procedure can be performed
on e.g. mobile devices.

On the whole, the paper is easy to follow and is generally well-written with only a few mistakes.
   
## Weaknesses
My main concerns are regarding some of the experimental procedures and results and the way they are reported.
- What bit strings were encoded for the experimental results? Was the bit string fixed for each image, or was it randomly sampled for each image separately? Are there any bit strings that are harder to encode than others, e.g. all 0 or all 1s? Intuitively, I do not expect it to be the case, but I wonder if the authors carried out any experiments.
- What hardware was used for the encoding is unspecified, which makes the reported decoding times less useful. It would be very good if the authors could specify if the encoding was performed on a GPU and if yes then how long it takes on a CPU and maybe even mobile devices. In a similar vein, the actual number of bits encoded is also unspecified. It would be good if the authors could maybe investigate how long the compression takes per pixel on average with error bars for example.
- As an extension of the second point, what is the biggest message the authors experimented with encoding?
- It is unclear to what the sentence at the top of page 8 is referring. The claim is that FNNS achieves 0% error at 2bpp with a 0.5% detection rate,
  but Table 5 doesn't seem to bear this out for any of the datasets. Is this sentence misplaced perhaps?
   
## Questions
   - As I understand, the two sources of decoding errors are clipping the difference (line 6 in Alg 1) and clipping the addition of the delta (line 7 in Alg 1).
     Did the authors investigate which of these steps tends to cause more errors? Could the issue be partially resolved by first reparameterizing the image $X \in [0, 1]^{W \times H \times 3}$
     to $ Z \in R^{W \times H \times 3}$ by applying an inverse sigmoid transform $Z = \sigma^{-1}(X)$ and optimizing $Z$ instead of $X$? This would ensure that the second clipping operation never needs to be performed, right?
   - This might be a simple point, but I wonder how robust the encoding is to the random seed used in FNNS-R to sample the NN weights in the following sense.
     Imagine the message is encoded using weights $w_1$ sampled randomly with seed $s_1$. What is the decoding error rate with weights $w_2$ sampled with $s_2$? Assuming that the
     NN predictions are iid Bernoulli $1/2$ variables, the decoding error for a random bit string should be around 50%. Is this indeed a correct intuition?

## After author rebuttal

The authors have addressed my concerns satisfactorily, and I believe that the final version of the paper will be a strong contribution to the literature.
 
summary_of_the_review: The core idea presented in the paper is simple and quite general, going beyond image steganography. The authors heavily outperform current steganography algorithms with their method and the community would benefit from its publication.

However, the article has some weaknesses that if the authors can address I will be happy to raise my score.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a method for image steganography based on adversarial attacks. Unlike previous works such as SteganoGAN, the proposed method fixes the decoder and performs an inference-time attack to produce a perturbation which maps to the correct message. Empirical results show the proposed method out-performs existing encoder decoder based design, and is also more robust in out-of-domain applications. 
main_review: Strengths: 
1. The paper is written clearly and easy to follow. 
2. The observations regarding the limitations of an encoder - decoder architecture is novel and provides new insights to the field. 
3. The empirical performance of the method is strong. The out-of-domain example is convincing and the decode accuracy of the method is very high. Moreover, the paper deals with the setting of an arbitrary binary message. 

Weakness:
1. The parts regarding JPEG compression can be expanded. For example, 
(a) what are the JPEG compression quality factors used for the experiment? 
(b) It would be interesting to combine the approach with JPEG-resistant adversarial attacks. But it's understandable if this is outside the current scope of the paper.  
summary_of_the_review: Overall I enjoyed reading this work. See main review for details.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This submission deals with steganographic paradigm : hiding a secret message in a cover content while remaining undetectable. The authors use a purely ML-based pipeline. They intend to achieve zero message-retrieval-error (which is normally a must-have for steganographic approaches). They use a neural network decoder and exploit adversarial perturbations to compute the stego version of the cover image. The loss from which gradients originate is the sum of binary cross-entropies between decoder output and message to embed. 

main_review: There has been a form of misunderstanding in some recent publications in the ML community that claim to do steganography. Stricto sensu, these papers do hide information in a cover content and this secret message is typically not visible to the human eye. Yet the problem is that those techniques are easily detectable by SOTA opponents (steganalysis algorithms) and thus one does not reach the required level of secrecy in this multimedia security context. Also, in practice, it is unthinkable to use an algorithm that cannot embed an encrypted message which means that the message must be recovered with 0% error all the time. This is why the steganography community has been working on the problem of minimizing detectability given a message length to embed and does not allow to trade message retrieval exactness with detectability. 
Fortunately, the present paper seems to be aware of the problem and attempts to remediate this issue by achieving 0% error using ML-based message embedding. 

There are a number of claims in the introduction with which I do not agree :
- Steganography is presented as a superset of other problems such as watermarking or copyright certification. I argue that these are separate problems with separate solutions even if they borrow some ideas from one another. This is again about the misunderstanding mentioned above.
- Classic steganography techniques « can only hide up to 0.4 bits per pixel  » in order to avoid detection. First, such an empirical result is highly dependent on the cover image. There are some for which one can embed much more and other possibly less. So this must be an average bpp. More importantly, I do no think any steganographic scheme can claim to achieve 0% detection with 0.4 bpp. The usual experimental setup is to provide a curve showing the detection rate as a function of bpp payload. This curve will be dependent on the image dataset and the chosen opponent (usually a SOTA steganalyzer). 
- StegoGAN is not SOTA steganography.. it might be ML-based SOTA steganography. But the mainstream steganography does better than this notably regarding detectability.

Concerning related works and novelty, the paper is not the first to exploit adversarial image to improve steganography see for example:

[1] W. Tang, B. Li, S. Tan, M. Barni, and J. Huang. 2019. CNN-Based Adversarial Embedding for Image Steganography. IEEE Transactions on Information Forensics andSecurity14,8(2019),2074–2087
 
[2] Bernard, S., Bas, P., Pevný, T., & Klein, J. (2021, June). Optimizing Additive Approximations of Non-Additive Distortion Functions. In Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security (pp. 105-112).


The authors use clipping to prevent the stego and cover to have too high pixelwise differences and to prevent out-of-range pixel value assignments.. In practice, most image formats use 1 byte for each pixel (or 3 bytes for RGB). So it would have been more convincing to clip image between 0 and 255 and then clip modifications to integers values. 

The quality of a steganographer should not be evaluated through reconstruction errors such as PSNR, MSE or SSIM because they do not say much about detectability. This is what Table 5 reveals. The detection rates are too high to entrust the algorithm for achieving secret communication. BTW, detection rate is a bit ambiguous term, it is true positive rate ? It would be better to show classifier error rate. 
The security of the steganography must be evaluated through SOTA steganalyzer error rates. The authors use ref. [You et al. 2020] as SOTA steganalyzer. Others are:

[3] Boroumand, M., Chen, M., & Fridrich, J. (2018). Deep residual network for steganalysis of digital images. IEEE Transactions on Information Forensics and Security, 14(5), 1181-1193. 

[4] Xu, G. (2017, June). Deep convolutional neural network to detect J-UNIWARD. In Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security (pp. 67-73).


The authors say on page 6, that they include some comparison with HUGO but I could not find it in the mentioned table (maybe I missed something). As a baseline, I think the authors should at least compare themselves to UNIWARD [Holub et al. 2014] although there are now more secure steganography schemes in the mainstream steganography literature. 

Besides, one of the main weakness of the presented study is that the achieved zero error for message retrieval is only empirical while with statistical image steganography zero error is a built-in property. So in practice, one has to trust the generalization performances of the model in order to send an encrypted message.

While section 6 is somewhat interesting, I think this use case is quite narrow. The security relies on 0% error + encryption but many other concurrent approaches can do that relying on the cover produced by the face GAN. So why the proposed approach should be used ? It is because the message to embed is large ? I agree that, here, detectability is not the top priority. But I am unsure about the interest of publicly sharing someone’s picture with the face of someone else.  If two parties want to share pictures they can use private messaging apps with encryption.
summary_of_the_review: pros : 
- it is clever to rely on adversarial examples and iterating to obtain a stego image

cons : 
- the evaluation in terms of undetectability (which is what matters most in steganography) is insufficient: one should provide the error rate of a few SOTA opponents. The method must also be compared to gold standards steganographers. 
- the zero error of message retrieval is only empirically checked and one cannot hope for perfect generalization performances
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work proposes Fixed Neural Network Steganography (FNNS), an algorithm for steganography leveraging adversarial attacks to hide a secret message in an image. The authors propose three variants of FNNS, show its performance under steganalysis, and propose face anonymization as a possible use case of FNNS. 
main_review: # Strong Points
* FNNS constitute a novel approach in the field of steganography leveraging adversarial attack techniques to hide a secret message in an image. 
* The paper is well written and easy to follow
* The presented application of face anonymization is interesting.
* The authors present that FNNS is capable of evading steganalysis frameworks. 

# Weak Points
* I am wondering about the computation time of the proposed approach? It appears that FNNS has to process each image individually with L-BFGS, which might be relatively time-intensive. On the other hand, previous techniques can hide secret information in a matter of a single forward pass. This might make FNNS inefficient at deployment in real-world scenarios.
* I am wondering about the motivation of the authors to choose the L-BFGS algorithm? In the adversarial attack community many other, maybe better or faster adversarial attack algorithms had been proposed. Did the authors experiment with other attack algorithms such as PGD, DeepFool, or C&W?
* The paper misses related works. Just to mention a few: [1,2,3,4,5,6,7,8], which should be discussed and ideally differentiated and compared to. 
* The source code is not provided? Will the code be publicly available? 

[1] Light field messaging with deep photographic steganography; CVPR 2019  
[2] Stegastamp: Invisible hyperlinks in physical photographs; CVPR 2020  
[3] UDH: Universal Deep Hiding for Steganography, Watermarking, and Light Field Messaging - NeurIPS 2020  
[4] HiNet: Deep Image Hiding by Invertible Network; ICCV 2021  
[5] Large-Capacity Image Steganography Based on Invertible Neural Networks; CVPR 2021  
[6] Deep Multi-Image Steganography with Private Keys; 2021  
[7] Enhancing the Security of Deep Learning Steganography via Adversarial Examples; Mathematic 2020  

# Additional Questions 
* The motivation for the naming of FNNS is not entirely clear to me. What does the **Fixed** in FNNS stand for? Aren’t all other neural network-based steganography techniques using fixed neural networks? 

summary_of_the_review: In my opinion, FNNS can provide a contribution to the steganography community. The proposed algorithm is novel and includes sufficient results and the evaluation and analysis appear valid. The authors also show an interesting use case of FNNS. However, the paper still has some weak points the authors should address to increase the quality and completeness of the paper. 

====== Edit November 23 ======  
Raised my score to Accept after reading the author's response.
review_title: null

Meta-review: This submission proposes a method for steganography, i.e. hiding "secret messages" in images. Specifically, the proposed approach implements a procedure similar to adversarial example generation, where a perturbation is found that a) is imperceptible and b) can be decoded by a fixed decoder. This approach results in the ability to hide a significant amount of information (up to 3 bpp) with essentially no decoding errors. While the resulting perturbations are sometimes detectable by existing methods, the authors nevertheless provide a compelling case that their approach is a significant step forward and demonstrate an interesting new application to face hiding. Reviewers generally found the paper easy to follow and clear and felt the proposed approach provided a new and effective way to perform steganography. However, there were many requests for clarification of the work's situation in the larger literature of steganography and adversarial ML. The authors have clarified their contribution in the rebuttal, and as long as these changes have made it back into the paper, I am happy to recommend acceptance.
Decision: Accept (Poster)