Paper: How Robust Are Energy-Based Models Trained With Equilibrium Propagation? (ICLR-2024)

Reviewer comment (Reproducibility):
The description of how to attack EBM is not detailed enough. If the adversary knows that EBM is used, the adversary will not try using the PGD attack on the traditional loss terms. (If PGD on the traditional loss terms is used for the experiment, it will damage the soundness of this paper.) What is the loss term that is used?

Author response:
We agree with the reviewer that it is unclear what loss function is used to train and attack the EP model, and that this information is important for accurately assessing the robustness of EP to adversarial attacks. Cross-entropy loss is used to train and attack all models tested here, including EP. As a result, we believe the experiments are sound because we are training and attacking EP the same way as all other models, which is the standard way to train and attack deep neural network classifiers in general. We will add these details in the main text to ensure that this is clear.

Reviewer comment (Reproducibility):
EBM itself should be described in more detail. How do EBMs make predictions? This detail is needed to understand how to perform gradient-based adversarial attacks on EBM.

Author response:
Describing EBM in detail
    - We are including clarifying sections in the methods section which explains how predictions are made by models trained with Equilibrium Propagation.
    - The training cost of such models are discussed in section 4.2. As of the time of writing this paper, EP has only been able to scale to ImageNet32 [R1] and takes exponentially more time and computational resources to attack, when implemented on standard GPU based hardware. Our implementations offer a speedup of 30% as discussed in section 4.2, which was one of the major reasons we were able to carry out gradient-based attacks in a reasonable amount of time.

Reviewer comment (Experiments):
How costly are EBMs compared to the traditional models? This information is needed to justify the simple model architecture used in experiments.

Author response:
The training cost of such models are discussed in section 4.2. As of the time of writing this paper, EP has only been able to scale to ImageNet32 [R1] and takes exponentially more time and computational resources to attack, when implemented on standard GPU based hardware. Our implementations offer a speedup of 30% as discussed in section 4.2, which was one of the major reasons we were able to carry out gradient-based attacks in a reasonable amount of time.

Reviewer comment (Experiments):
The model architecture (four convolutional layers followed by a fully connected layer) is too simple. Is this because of the expensive computation cost of EBM? How do you know that the findings will generalize to more complex architecture?

Author response:
The implemented model architecture is kept simple because it is expensive to train on standard hardware. As the world starts moving towards non-standard spiking hardware, such models will become the more efficient alternative. We are confident that these would scale to more complex architectures / problems as previous work by Laborieux et al. [R1] have shown that even such small models perform well across various complex tasks (CIFAR 10, CIFAR100, and ImageNet-32).

Reviewer comment (Evaluation):
How do you measure the severity of each natural noise? The authors should describe how to measure the severity of each type of natural noise.

Author response:
Severity of natural noise is described in the paper that introduced the CIFAR10-C dataset [R2], which is a standard benchmark used to evaluate robustness to natural noise.

Reviewer comment (Experiments):
Figure 5 might not be enough to conclude that attacks on EP-CNN are more semantic. (Some people may not agree that the “deer” is induced by the attack.) Probably, it would be better to use a dataset where such semantic changes are more visible. For example, changes from attacking EP-CNN trained on MNIST or a dataset containing face images might be more illustrative.

Author response:
We admit to the fact that attacked images in Figure 5 may not always appear as “deer” to our readers. To this end, we are including a collage of images from multiple classes bearing perturbations in the appendix, to convince our readers that perturbations in EP are not limited to the background.

Reviewer comment (Presentation):
To add an appendix, you should use `\appendix` command in LaTeX.

Author response:
We apologize for the use of sections instead of using /appendix in the paper. We thank reviewer GSeC for pointing out this error. We have fixed this issue in our paper.

Reviewer comment (Experiments):
Though the paper mentions that the high time cost limits EP models to relatively shallow architectures, I think the experiments that mainly focus on PGD are not enough. There are other attack methods, e.g., FGSM [1], C&W [2], AA [3], etc. The conclusion that EBMs trained with EP have better robustness will be more convincing if more attack methods are evaluated, even though the models have relatively simple architecture and the datasets are relatively small.

Author response:
We have indeed conducted the AA attack and presented our results in Figure 4 in the original submission. We also conducted C&W attacks across all models and have added the results in the appendix. However, we would like to discuss a few details on FGSM and C&W.
   - FGSM is a 1-step version of PGD and thus superseded by both PGD and AA, which we use in other experiments and have presented in the results section of the original submission. Many other previous works have also illustrated that PGD attacks provide a much better test of a model’s robustness relative to FGSM. For all of these reasons, we believe it is unnecessary to include FGSM attacks in our experiments.
    - We agree with the reviewer that the inclusion of C&W attacks would provide a more thorough evaluation of EP’s robustness, and we thank the reviewer for this suggestion. We have included results from a C&W attack in the modified submission. Under these C&W attacks, EP is one of the most robust models, even more than the adversarially trained models, for the CIFAR-100 dataset.

Reviewer comment (Theory):
The superiority of the EP models in robustness is mainly shown by experimental results, which lack theoretical analysis and support. If rigorous mathematical analysis is too hard, then at least an intuitive analysis based on formulas that show why EP models can function well should be provided.

Author response:
We appreciate your comment about our work lacking theoretical analysis. We added a section in the appendix that provides intuitive analysis and hypothesis of why EBM models are more resistant to adversarial attacks as opposed to traditional training methods and we leave a thorough theoretical approach for future work.

Reviewer comment (Theory):
For the second term on the right side of Equation 1, I think it is possible that there are pooling layers following the FC layers. In other words, the second term can still contain $\mathcal{P}(\cdot)$?

Author response:
There are no pooling layers following the fully connected layers, so the second term on the right side does not contain any pooling terms.

Reviewer comment (Theory):
According to my understanding, in Section 3.1, the training goal is to update the convolution weights $w_{n}$? However, I am unclear how the weights $w_n$ are updated based on Equations 4-6 related to the states $s^{n-1}$ and $s^{n+1}$. Can there be more explanation?

Author response:
Weight update rules are provided in the supplementary section of the original paper[R1].

Reviewer comment (Reproducibility):
As mentioned in the limitation, ``EP is relatively sensitive to the hyperparameters used to train the model as well as the seed used to initialize the weights``. Are there some heuristic rules to set the hyperparameters and the weights' initialization?

Author response:
For a hyperparameter search, one has to check the state of the system after activations have been applied and make sure the value of the states post-activation are not zeroed.

Reviewer comment (Writing):
Some editorial issues (just listed a few):
- Better not to use contractions such as ``didn't``, ``don't``, ``weren't``.
- It is a bit strange that there are the right brackets after BP-CNN and AdvCNN, respectively, but not the right bracket after ViT in Section 4.1.1.
- A missing period ``.`` after the second paragraph in Section 4.1.1.
- Inconsistent representations when referring to figures. For example, ``Fig. 2 and Fig. 10`` and ``Figure 1`` on page 8.
- On page 6, ``Figure 3,4`` should be ``Figures 3,4``.

Author response:
We are very grateful to you for pointing out the editorial issues and we have made sure that the paper is grammatically consistent and easier to read.