Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a method which combines the benefits of iterative optimization and a learned encoder-decoder for the task of image steganography. Similar to the style of LISTA, the paper learns individual optimization blocks in an E2E fashion while applying the blocks iteratively at inference time. Experiment results show superior performance in terms of error rate and PSNR compared to previous baselines. 


strength_and_weaknesses: Strengths: 
* The paper is well-written and easy to follow. 
* The proposed method is intuitive and novel. The rationale behind applying learnable blocks iteratively is explained clearly. 
* The experiment results are strong and convincing. 

Weaknesses. 
Minor points: 
* Figure 1 could include a scaled version of the diff if space allows. 
* Figure 4 CelebA sample has a very visible color shift a 1bit per pixel. Given the relatively high average PSNR reported, are the shown results correct or an outlier in the dataset? From the supplementary material samples, other images do not show this color shift.

clarity,_quality,_novelty_and_reproducibility: Clarity: Good. 
Quality: Good.
Novelty: Good. 
Reproducibility: The authors have not claimed to release code. 
summary_of_the_review: Overall I believe this paper has enough novelty and strong enough experiment results to recommend acceptance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a method called Learned Iterative Steganography Optimization (LISO) that performs image steganography, i.e. embedding arbitrary information in image data (not metadata) in a way that is difficult to see or detect through statistical analysis. LISO uses iterative optimization based on a gated recurrent unit (GRU) to determine how to perturb image pixels to minimize visibility and maximize recovery of an arbitrary message. The authors show that compared to existing steganography methods, LISO is fast (Table 4), leads to low error rates in recovering the hidden message (Table 1), and can be adapted to more resilient to JPEG compression and more resistant to automated detection.

strength_and_weaknesses: The primary strength of the paper is the interesting approach and solid empirical results. The approach is based on iterative optimization using GRU where each iteration takes the previous estimate of the perturbation and a gradient, and the hidden state allows the method to minimize the loss more quickly and more effectively than methods that only take a gradient (see Fig 3).

The empirical evaluation involves three components: the error rate in recovering the hidden message, the image quality after embedding the message, and runtime. Table 1 shows that LISO (coupled with L-BFGS) outperforms SteganoGAN by a significant margin. Compared to FNNS-D, both methods reach zero rates (or very close to it) up to 3 bits per pixel (bpp) but only LISO has near-zero error rates at 4bpp. On the other hand, FNNS-D does have higher PSNR and SSIM for two out of the three data sets (MS-COCO and CelebA but not Div2k). It's not clear to me how big of a problem this is, though, since LISO uses a GAN-like loss to keep perturbed images near the manifold of natural images. I would expect this loss to make the images look more realistic at the expense of fidelity (this is often discussed as high perceptual quality but also high distortion). Since PSNR and SSIM are more closely aligned with distortion than perception/realism, they may not be good measures of image quality, or, at least, they're at odds with the GAN-like loss used by LISO (see work on the perception-distortion tradeoff by Blau & Michaeli for more information).

Furthermore, LISO runs much more quickly than FNNS-D and LISO+L-BFGS has a significant speed advantage over FNNS-D too.

In practice, the main weakness of LISO is that performance drops with lossy compression, however LISO-JPEG (a variant of LISO that incorporates JPEG compression into the training procedure) fairs better than the baseline methods. Since natural images are almost always compressed using a lossy codec, the main results ("LISO-PNG" that uses *lossless* compression) has relatively little real-world impact.

clarity,_quality,_novelty_and_reproducibility: The paper is clear and the presentation and empirical evaluation are of high quality. I have not previously seen GRU (or other RNNs) used for iterative optimization so I believe this approach is novel.

Although the paper is conceptually clear, releasing code would improve reproducibility considerably since there may be many details that effect performance in how the GRU-based optimization and GAN-loss are implemented.
summary_of_the_review: I'm recommending that this paper be accepted based on the interesting use of GRU for fast (low iteration count) optimization, and the strong empirical results presented.

Initially, I wasn't sure that steganography applications were a good fit for ICLR, at least without further analysis of the actual representation that's learned, but the main baseline (FNNS) was published at ICLR 2021. This work appears to be an interesting and significant improvement over FNNS.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a new method for image steganography based on an iterative optimisation method (gradient-descendent) able to achieve zero recovery error rate for encodings up to 3bpp. The method is able to keep steganography output images close to (or on) the manifold created by natural images. The experimental results show that the paper outperforms the state-of-the-art in terms of recovery (decoding) error rate and in terms of computational cost.
strength_and_weaknesses: 
Strengths:

1) the paper is well written and it is very clear
2) the novel ideas of the paper explores two separate previous approaches to the problem, namely the neural learning and the iterative optimisation. The method presented by the paper is able to jointly explore these two approaches by creating a learning process for the optimisation problem, able to learn the gradient descent direction.
3) the experiments are thorough and show that the method outperforms the state-of-the-art

Weaknesses:
1) when comparing the method and the cited methods of the state-of-the-art, the authors usually state the recovery error rate of the latter ones, by 5-25% for 4bpp, and state that the presented method can achieve 0% recovery error rate for up to 3bpp. In my opinion, it would be more fair and clear to compare in the text both the presented method and SOTA in both 3bpp and 4bpp. notice that the results are stated in Table 1, but the text is not so clear mentioning it.
2) the paper does not explores sufficiently the resulting image quality in terms of being natural (indirectly measured by PNSR and SSIM). The results of Table 1 show that the proposed method does not always achieve better results in terms of image quality, so I suggest a better discussion in terms of image quality and maybe including some subjective evaluation of images by humans.
3) In section 4, in the Training paragraph, it is not very clear whether the end-to-end training is image-specific or uses several different images. 
4) There are a few typos. I suggest a carefully revision of the text.

clarity,_quality,_novelty_and_reproducibility: The paper is overall very clear and presents novel and interesting ideas on the problem of steganography.
The method presented by this paper seems to be reproducible.

summary_of_the_review: The paper is interesting and well written. It would benefit of some improvements in terms of comparison of the recovery error rate and, mainly, in terms of resulting image quality.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work proposes Learned Iterative Steganography Optimization (LISO), a steganography approach that leverages deep neural networks and iterative optimization to hide a secret message in an image. LISO is faster than previous optimization-based approaches and has low message recovery error. 
strength_and_weaknesses: ## Strengths 
(+) Deep-learning-based steganography approaches are an active research field, gaining more and more popularity. This work is one of a few, that investigates the optimization-based approach.  
(+) This work presents convincing results, with a relatively low error rate and average time (outperforming FNNS).  
(+) I highly appreciate the analysis regarding steganalysis, runtime, and JPEG compression.  

## Weaknesses 
(-) I might have missed it, but I would be interested regarding the hidden messages. Were the bit string messages chosen randomly? I am also interested if the authors encountered “adversarial” bit strings, meaning bit strings that are difficult to encode.  
(-) I appreciate the evaluation in Appendix I. What are the limitations of the algorithm regarding the length of bit strings? Or in other words, at what payload lengths, does the algorithm not produce sufficient results?  
(-) I am curious about a study regarding the cover image. Are there cover images, for which it is easier or more difficult to encode messages, e.g. random noise, black-and-white images, or single-colored images?   
(-) The authors strongly argue that “the steganographic image remains on the natural image manifold with the help of a critic”. This argument is repeated multiple times throughout the paper, but no concrete evidence or proof is provided. Hence, this claim remains an assumption.  
(-) Will the code be provided? 

clarity,_quality,_novelty_and_reproducibility: ## Clarity
This work clearly written and technically sound to the best of my judgment.

## Quality 
The paper quality is sufficient in my opinion. 

## Novelty
In my judgment, this work holds novelty in terms of the proposed approach and the accompanying analysis and results. 

## Reproducibility
Since the code is not provided, it might be difficult to reproduce this work. 
summary_of_the_review: The presented LISO is a novel steganography approach that outperforms previous optimization-based steganography approaches. The authors provide convincing results and analysis regarding their approach. From my judgment, this work holds sufficient novelty for a subgroup in the ML community.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Thanks for your submission to ICLR.  The reviewers were generally positive about this paper, with all four reviewers recommending to accept.  On the positive side, the reviewers liked the clarity of the paper/writing, the novelty of ideas, and the thorough experimentation.  They all mentioned weaknesses as well, but these were all minor.
justification_for_why_not_higher_score: Two of the four reviewers were a bit less enthusiastic, with scores leaning accept.  All four reviewers found minor weaknesses in the paper.
justification_for_why_not_lower_score: All four reviewers ultimately are supporting of the paper being published, so there is consensus on this being accepted to the conference.
Decision: Accept: poster