From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: A well-known NP-hard problem is graph matching, GM (as opposed to the polynomial-time solvable matching problem in a single graph): matching between two---or more---given graphs in order to maximize an affinity score of the matching. This work considers the (probabilistic) robustness of algorithms for GM algorithms against norm-bounded adversarial perturbations; it develops new algorithms for randomized “smoothing”. The robust matching problem on the graph is decomposed into subproblems defined over the nodes; the noise for building the smoothed classifier is sampled from a joint Gaussian that is defined to capture correlations between the different nodes. Experimental evaluation is performed on a popular dataset and is generally promising (e.g., can certify larger radii than could earlier approaches). The paper is solid overall.

The setup needs to be motivated better: while robustness is critical in some applications (e.g., vision in self-driving cars), the security implications of non-robust GM needs to be described in more depth. Furthermore, randomized smoothing makes inference slow, and needs to be motivated better for  certified GM robustness. Even the work of references [13,14] in the paper is applicable as far as the theory goes (although scalability is an issue): there appears to be no bottleneck in GM that will stymie [13,14] from inferring correlations between nodes. Also, a stronger case can be made that a smoothed classifier can solve security problems when safety is critical. 

justification_for_why_not_higher_score: The utility of robustness for the studied graph-matching problem needs to be made clearer (as compared to obvious settings such as vision in self-driving vehicles); the fact that inference gets slowed down also has to be addressed. 
justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Could you start to discuss?
comment: Dear Area Chair,

Would you mind to start the discussion for our paper (it not done yet)?  It would be greatly appreciated if we have the chance to respond your further questions after your internal discussion.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Summary of the whole talking between reviewers and authors
comment: Dear Area Chair and Reviewers,

We would like to express our sincere thanks to the four reviewers for their time and valuable suggestions. In the rebuttal, we have elaborated our explanations, included more experiments, and revised our paper following all reviewers’ suggestions.

Based on reviewers' prompt feedbacks, we are encouraged that the concerns of reviewers BhnR, n9K9, Sgyn have been mostly addressed, and reviewer BhnR improved the score (from 5 to 6).

We hope and think reviewer 7S16 also acknowledged our response, except for a debatable dispute (which we really appreciate) on the motivation of this paper. Meanwhile, we note that reviewers BhnR and Sgyn showed satisfaction to our further explanation of our motivation, in our response.

Below we try to summarize the rebuttal and hope it could facilitate your final evaluation.

* We explain and supplement the motivation of this work (Reviewer BhnR, 7S16), including the significance and impact of the robustness certification for the GM solver, which is acknowledged by the reviewers.
* We explain why we implement the robustness certification for GM solver based on the randomized smoothing framework (Reviewer Sgyn, 7S16). **First**, we explain the significant differences between this work and the previous randomized smoothing framework (Reviewer Sgyn), and we get a positive reply from the reviewer Sgyn. **Second**, we compare the pros/cons of the randomized smoothing framework and other methods for certifiable robustness, arguing that the randomized smoothing framework is suitable for certifying the robustness for GM solvers and it has the potential for direct extension to other combinatorial optimization problems (Reviewer 7S16) -- e.g. for node labeling based solvers: ["Learning Combinatorial Node Labeling Algorithms"](https://arxiv.org/abs/2106.03594). **Third**, as for **the concern by reviewer 7S16** whether randomized smoothing (RS) is suitable for mission-critical applications, we also give a detailed response. Reviewer 7S16 thought that RS is a probabilistic framework and cannot ensure zero mistake and therefore may not be suitable for mission-critical applications. We have to say we do not fully agree and this dispute may go beyond the technical part of this paper: it is more about the applicability of RS itself which has been well acknowledged and used in the community. In fact, we are in the optimistic side and RS is exactly a mathematical tool to endow a machine learning module a degree (lower bound) of certification (up to infinitely close to 100% correctness by setting a small enough $\alpha$). We also fully respect reviewer's 7S16's rigorousness and modify the wording "mission-critical" to "near-mission critical". Moreover, we believe there are still broad areas using our technique, especially with the human-specified $\alpha$ to achieve almost controllable guarantees.
* We also feel inspirational by the comments of Reviewer n9K9 and 7S16 on better certifying the overall robustness of solution matrix $X$. We supplement with further techniques (including definitions and experiments) to make the paper more comprehensive and pertinent.
* In terms of experimental evaluation, **first** we adopt reviewer BhnR's suggestion and introduce Average Certified Radius (ACR) to quantize the performance. **Second**, we add relationship tables of accuracy (accuracy of base solvers, standard accuracy and certified accuracy of certified solvers) and radius to show the impact of certified robustness on the matching accuracy of the solvers based on the suggestion of reviewer 7S16. **Third**, we perform a study on the impact of different choices for constructing $\mathbf{\Sigma}$ on the certified robustness based on the suggestion of reviewer 7S16. The above experiments are a good supplement to our work, and the experimental results have also been recognized by the reviewers (BhnR and 7S16). **Fourth**, we also explain the experimental protocol in more detail (Reviwer BhnR), and have made modifications to the presentation of the paper (Reviewer 7S16, Sgyn, n9K9).

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_BhnR
title: Thanks
comment: Thanks, this helps highlight the effectiveness of your method.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Looking forward your reply to our strengthened clarification on motivation (1/2)
comment: We appreciate the prompt reply and helpful suggestions. We have further clarified the motivations of random smoothing as follows. We sincerely hope you are satisfied with our response, and could provide us further instructions if necessary. 

We agree that the randomized smoothing [9] method may not provide 100% correct certification guarantee for strictly mission-critical tasks (e.g. space exploration). However, we would like to elaborate on our understanding of the robustness on some near-mission critical scenarios in one's daily life. 
* First, as described in works [19-23], deep learning has achieved tremendous success over the years, and it starts to be deployed in various real-world scenes, including some near-mission critical applications (e.g. autonomous driving, face recognition in ATMs). Meanwhile, deep learning approaches have been shown inevitably vulnerable to adversarial attacks [20-23]. As a result, the study of robustness against adversarial attacks has drawn intensive attention in recent years. Currently, typical robustness-based methods include adversarial training [24,25], regularization techniques, certified defenses [9,13,14,28-30], etc. Although these methods have their own strengths and weaknesses, they are all promising researches to promote the more reliable deployment of deep learning models in real-world near-mission critical applications [19]. 
* Second, we would like to clarify that, in near-mission critical applications, 100% correct guarantee is not a must though ideally preferable. Actually, a robustness method is also desirable in practice, as long as it can provide a provable guarantee to control the uncertainty within a certain high confidence interval. For example, as shown in industry standard of autonomous driving [26,27], it usually involves a comprehensive safety assessment indicator which indeed permits different confidence levels of safety assurances. In such near-mission critical scenes, we believe a randomized smoothing method can well satisfy the robustness requirement with probabilistic and controllable guarantees. Therefore, we modify our description "mission-critical" to "near-mission critical". 
* Third, we would like to highlight again that compared with other defense methods, certified defenses can further guarantee the robustness of the network under any type of attacks (even unknown ones). Bound propagation [13,14] and randomized smoothing [9] are both representative methods of certified defenses. Bound propagation is a deterministic certified defense method, yet unfortunately, it has difficulties in scaling to large models. Recently, randomized smoothing has emerged as a certification approach from the probabilistic paradigm that scales well to large models and datasets. Since the model for graph matching is relatively large, only the randomized smoothing can be used to certify its robustness at present. Moreover, there are two other distinct advantages of applying randomized smoothing to graph matching. One is that randomized smoothing [9] treats all functions (regardless of deep learning or non-learning based graph matching functions) as black-box models, thus it has the potential for direct extension to other combinatorial optimization problems. The other is that graph structure as a whole conveys important information, so we use a smoothing distribution $\Sigma$ to construct correlations between nodes.

To sum up, we discuss the robustness of near-mission critical scenarios as well as the strengths and weaknesses of randomized smoothing [9] and bound propagation [13,14] certification methods. (We believe they are orthogonal and supplementary to each other.) However, detailed comparison and justification for these two categories of works are actually beyond the scope of our rebuttal, and they are not the topics of our paper.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Looking forward your reply to our strengthened clarification on motivation (2/2)
comment: In addition,
* As for the " offline black-box verification " you mentioned, our method regards the model as a black box during certification, which can realize this application. Following this suggestion, we will include this important application in our final revision.
* As for the " work of https://arxiv.org/pdf/2203.03153.pdf " you mentioned, they develop a framework for verifying the properties of GNN-based job schedulers. They verify GNNs including message-passing components, but the graph matching model is not just composed of GNNs. And although this method has generality and can be used to verify robustness on image classifiers as mentioned in the paper, they only experiment with fully-connected networks on MNIST dataset and the percentage of solved properties (a performance evaluation index) is even far from [14]. Therefore, like [13,14], it cannot be used to solve the robustness certification of large-scale graph matching models.


[19] Silva S H, Najafirad P. Opportunities and challenges in deep learning adversarial robustness: A survey[J]. arXiv preprint arXiv:2007.00753, 2020.\
[20] Akhtar N, Mian A, Kardan N, et al. Advances in adversarial attacks and defenses in computer vision: A survey[J]. IEEE Access, 2021, 9: 155161-155196.\
[21] Xu H, Ma Y, Liu H C, et al. Adversarial attacks and defenses in images, graphs and text: A review[J]. International Journal of Automation and Computing, 2020, 17(2): 151-178.\
[22] Li Y, Cheng M, Hsieh C J, et al. A Review of Adversarial Attack and Defense for Classification Methods[J]. The American Statistician, 2022: 1-17.\
[23] Yuan X, He P, Zhu Q, et al. Adversarial examples: Attacks and defenses for deep learning[J]. IEEE transactions on neural networks and learning systems, 2019, 30(9): 2805-2824.\
[24] Yan Z, Guo Y, Zhang C. Deep defense: Training dnns with improved adversarial robustness[J]. Advances in Neural Information Processing Systems, 2018, 31.\
[25] Ye S, Xu K, Liu S, et al. Adversarial robustness vs. model compression, or both?[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 111-120.\
[26] Salay R, Queiroz R, Czarnecki K. An analysis of ISO 26262: Using machine learning safely in automotive software[J]. arXiv preprint arXiv:1709.02435, 2017.\
[27] Chia W M D, Keoh S L, Goh C, et al. Risk Assessment Methodologies for Autonomous Driving: A Survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2022.\
[28] Rosenfeld E, Winston E, Ravikumar P, et al. Certified robustness to label-flipping attacks via randomized smoothing[C]//International Conference on Machine Learning. PMLR, 2020: 8230-8241.\
[29] Fischer M, Baader M, Vechev M. Certified defense to image transformations via randomized smoothing[J]. Advances in Neural Information Processing Systems, 2020, 33: 8404-8417.\
[30] Pérez J C, Alfarra M, Giancola S, et al. 3DeformRS: Certifying Spatial Deformations on Point Clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 15169-15179.

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_7S16
title: Motivation still needs work
comment: Dear Authors,

Thanks for the prompt reply. After reading the response, I am still not convinced as the sentence " hence, in theory, our method is highly reliable and can be applied to mission-critical applications" is misleading. Practically, zero uncertainty is not feasible, and therefore the use of such techniques for safety-critical applications (e.g., space exploration or autonomous driving) is not suitable. If one goes by only theory, then even the work of [13,14] is applicable to your networks which you rightly point out may not scale to the datasets considered here. There is also nothing inherent about the graph matching problem that will prevent the method of [13,14] to capture correlations between nodes. For example, the work of https://arxiv.org/pdf/2203.03153.pdf uses similar methods to verify GNN-based schedulers.

Overall, the work is promising but I believe the use-case scenario where randomized smoothing is useful for graph matching needs to be motivated better in the paper as it also increases the inference time of the model. The authors can try to focus on an offline black-box verification setting where an individual may want to compare their base models without sharing weights. But it's hard to be convinced that a smoothed classifier can solve security problems in safety-critical settings.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: New Response to Reviewer 7S16 (1/2)
comment: Thanks for your prompt feedback and we feel happy that we had made an improved version and explanation to ease some of your concerns. We also appreciate your new questions and suggestions，our responses are as follows.
>***Q1: For high-security applications and specifically mission-critical ones, one should avoid relying on a model that can probabilistically make mistakes by design (however unlikely it might be). In this sense, your certification does not solve a security problem. An application of randomized smoothing for graph matching could be that you can use your method to compare two base classifiers and select the one obtaining the best tradeoff between accuracy and robustness. I am not sure if the randomized smoothing presented here can be used for this.***
> 
**R1:** 
* **Randomized smoothing (Cohen et al.2019) [9] is probabilistic yet can be of arbitrary small uncertainty.** We agree that randomized smoothing is a probabilistic approach. In our method, the exact values of $p_{A}$ and $p_{B}$ in Section 4.2 cannot be obtained, and so we follow the Monte Carlo sampling method mentioned in [9] to get the lower bound of $p_{A}$ and upper bound of $p_{B}$. To evaluate the reliability of sampling, we also use hypothesis test [18] as in [9] by using $\alpha$ to represent the probability of getting incorrect matching results. This is the standard and general results under randomized smoothing. In this paper, we set $\alpha=0.001$, and so there is a high probability (99.9% in this paper) to ensure the certification. $\alpha$ can be set arbitrarily small hence in theory our method is highly reliable and can be applied to mission-critical applications. 
* **There are two important properties of randomized smoothing [9] that make it suitable for the robustness certification of graph matching.** **First**, randomized smoothing [9] treats the model as a black box, so it can be easily extended to more graph matching functions. **Second**, it is usable for large-scale models, which is also necessary for graph matching functions which need to process image inputs. However, other certification methods known currently cannot simultaneously satisfy the above two properties.
* As for Q1 mentioned about **applying our method to compare two different base functions**, actually we have already done such a comparative experiment in the paper. Our method evaluates the certified robustness of the smoothed function $g$ as in Eq. 4 rather than the base function $f$. In Figure 2, we compare the certified accuracy and robustness of four base graph matching functions (NGMv2, CIE-H, PCA-GM and GMN). Below we use a table to directly show the tradeoff between the certified accuracy and robustness of four different base functions using our method ($\sigma$=5) on Pascal VOC dataset.  The certified accuracy at $R = 0$ is just the standard accuracy of the smoothed function.

|  | STD.ACC(%) |CERT.ACC (%)R=1.0|CERT.ACC (%)R=3.5 |CERT.ACC (%)R=7.0 |CERT.ACC (%)R=10.5 |CERT.ACC (%)R=14.0 |
| --- |  --- | --- | --- | --- | --- |--- |
NGM2| **75.6** |**73.8**|**63.7** |**51.5** |**36.4**|**12.2**|
CIE-H| 57.8 |53.0|41.1 |22.9 |4.1 |0.0 |
PCA-GM| 56.2 |51.4|36.0 |21.2 |5.9 |0.0 |
GMN|40.2 |36.1|26.1 |17.2 |9.5 |1.5 |

$R$ in the table represents $\|\|\delta\|\|_{\text{lower}}$ calculated by Eq.9. The table shows that NGMv2 base function can get better tradeoff between the certified accuracy and robustness.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: New Response to Reviewer 7S16 (2/2)
comment: >***Q2:The work of [13,14] does not do interval bound propagation (IBP) but instead does relational reasoning and can capture correlations between different neurons/inputs as you also try to do so it may not be as imprecise as you think. You mention "but they have requirements on the network structure", can you elaborate on what specific requirements [13,14] have?***
>
**R2:**
* **You mention "[13,14] capture correlations between different neurons/inputs as you also try to do so it", however we respectfully disagree with this interpretation.** In fact, the correlation we mentioned is not the connection between neurons/inputs but the connection between nodes (in the set $\mathbf{z}^{1}$) in the graph structure, which is unique to graph matching problems. For example, on Pasacal VOC dataset nodes represent the locations of keypoints labeled by human or keypoint detectors, as the center of the circle in Fig.1(b). The graph structure as a whole carries important information, so we use a smoothing distribution $\Sigma$ which constructs correlations between nodes. 
* **We further explain our understanding of [13,14].** **First**, randomized smoothing [9] regards all functions (regardless deep and non-learning for graph matching functions) as black-box models, and have the potential for direct extension to other combinatorial optimization problems. In contrast, the method of [13,14] requires the structure and parameters of the model which can be less applicable for different models for certification. **Second**, as stated in [14], all BaB based verifiers (including [13,14]) are still limited to relatively small models far from those with high capacity trained at ImageNet scale. However, Pascal VOC used in this paper contains a large number of image pairs and needs high-capacity model, which are difficult to be certified by the techniques in [13,14] as they currently only support small-size models.

* In summary, we argue that various certified robustness methods have their own strengths and weaknesses. Considering the general applicability for different graph matching solvers, we chose the randomized smoothing method as our base method.

>***Q3:What is the accuracy of a base classifier without randomized smoothing? Providing this number will help the readers understand how much accuracy is being sacrificed for obtaining probabilistic robustness.***
>
**R3:** Thanks for your very valuable suggestion and the graph matching accuracy of the base matching function is supplemented as follows. We show the accuracy of $f$ (NGMv2 in the table), standard accuracy and certified accuracy of different certified radius $\|\|\delta\|\|_{\text{lower}}$ of four methods (when $\sigma$ = 5) on Pascal VOC dataset.  The certified accuracy at $R = 0$ is just the standard accuracy of the smoothed function, STD denotes standard: 
|  | F.ACC(%) |STD.ACC(%)|CERT.ACC (%)R=3.5 |CERT.ACC (%)R=7.0 |CERT.ACC (%)R=10.5 |CERT.ACC (%)R=14.0 |
| --- |  --- | --- | --- | --- | --- |---|
RS|77.3 |76.7 |**66.9**|0.0 |0.0 |0.0 |
DDRS|77.3 |**77.4** |66.6 |50.5 |18.2 |0.0 |
 ANCER| 77.3|76.5 |64.2 |49.1 |23.8 |7.8 |
  SCR-GM| 77.3|75.6 |63.7 |**51.5** |**36.4**|**12.2**|

We find that the accuracy sacrificed of the smoothed function $g$ is acceptable (see last row for our method: w/o RS: 77.3-> w/ RS: 75.6), which also supports our adopting of randomized smoothing [9] in the graph matching problem. 

We also find that the standard accuracy of the DDRS method is better than the accuracy of base function $f$. This may be because the labeling of nodes on the Pascal VOC dataset is not perfect suffering from its inherent vulnerability due to the randomness of human labeling or keypoint detectors as also argued in (Ren et al.2022) [4], and $g$ can expand the range of node options through perturbation $\varepsilon$ in Eq. 4 which partially compensates for the above imperfection. This is an extra bonus which inspires us that our ideas of constructing correlation of nodes may be used to improve the accuracy of base graph matching functions in the future. We also modify Table 2 in the paper.

[18] Hung K, Fithian W. Rank verification for exponential families[J]. The Annals of Statistics, 2019, 47(2): 758-782.

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_n9K9
title: Response
comment: I thank their authors for their response, clarifications, and edits to the paper, this has been helpful in answering my questions.

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_7S16
title: Follow up comments
comment: Dear Authors,

Thanks for your excellent response which addressed some of my concerns. However, I still feel that the reason for using randomized smoothing here is not convincing enough. In your response, you mention 

"Our certified algorithms are more suitable for applications with high-security requirements to ensure consistent prediction results with minimal loss of accuracy.  We hope our research can shed light on the mission-critical applications.", 

For high-security applications and specifically mission-critical ones, one should avoid relying on a model that can probabilistically make mistakes by design (however unlikely it might be). In this sense, your certification does not solve a security problem. An application of randomized smoothing for graph matching could be that you can use your method to compare two base classifiers and select the one obtaining the best tradeoff between accuracy and robustness. I am not sure if the randomized smoothing presented here can be used for this.

The work of [13,14] does not do interval bound propagation (IBP) but instead does relational reasoning and can capture correlations between different neurons/inputs as you also try to do so it may not be as imprecise as you think. You mention "but they have requirements on the network structure", can you elaborate on what specific requirements [13,14] have?

What is the accuracy of a base classifier without randomized smoothing? Providing this number will help the readers understand how much accuracy is being sacrificed for obtaining probabilistic robustness.

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_Sgyn
title: Reponse
comment: Thanks for the authors' responses. Thanks for the other reviewers' comments as well.

I think the authors' explanation about the impact and motivation of this work helps me to better evaluate this work. The general response and the responses to other reviewers are also very helpful.

The new figure is easier to understand.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: General Response
comment: Dear Area Chair and Reviewers,
We thank all reviewers for their constructive comments and have updated our paper accordingly. In our general response, we give quick response for two important aspects: i) significance and ii) our added new treatment on certifying the solution $X$ with structure (we currently put it in appendix while we can also move it to main paper upon request).

**1) Significance and impact (reviewer BhnR, 7S16):**
Foremost, we try to address and clarify some common concerns. In the paper, we propose a general framework for incorporating existing RS-based techniques for certifying graph matching solvers by converting the relaxed matching $\mathbf{S}$ (by an arbitrary relaxed GM solver) to node matching. We believe robustness certification for the GM solver is a good starting point for the more open problem w.r.t. certification for combinatorial solvers (see our detailed response in below for each reviewer).


**2) Certification on the structure of $X$  (reviewer n9K9, 7S16):**
We also feel inspirational by the valuable comments of Reviewer n9K9 (Q4) and 7S16 (Q8) to better certify the overall robustness of solution matrix $X$, and we did have already developed further technology in this regard and put the content in the Appendix E. (We are also happy to put it in the main text and update the main pdf upon your feedback).

- Definition of smoothed function for $X$
We define a certified robustness to the overall $X$, which means that when the perturbation to the input is within a certain range, the output $X$ matrix is guaranteed to remain constant with a confidence $c$. Consider a graph matching problem from input space to partial permutation matrices $\mathcal{X}$. As discussed above, randomized smoothing (RS) is a technique for constructing a smoothed function $g$ from an arbitrary base function $f$. When queried at the input $\left(\mathcal{G}\_{1},\mathcal{G}\_{2},\mathbf{z}^{1}, \mathbf{z}^{2}\right)$, the smoothed function $g$ returns whichever matrix $X$ the base function $f$ is most likely to return when $\mathbf{z}^{1}$ is perturbed by noise:
$$
g=\underset{X \in \mathcal{X}}{\arg \max } \mathbb{P}(f\left(\mathcal{G}\_{1},\mathcal{G}\_{2},\mathbf{z}^{1}+\varepsilon, \mathbf{z}^{2}\right)=X),\\
\text{where} \  \varepsilon \sim \mathcal{N}\left(0,\mathbf{\Sigma}\right).
$$
The distribution of additive noise $\varepsilon$ is a joint Gaussian distribution matrix whose variance $\mathbf{\Sigma}$ represents the correlations between nodes. In addition, $\mathbf{\Sigma}$ is a hyperparameter for certified function which controls the robustness/accuracy trade-off.
- Robustness Guarantee for $X$
We define a robustness guarantee with confidence $c \in [0,1]$, which ensures that the similarity between the output matrix calculated by $g$ and its ground truth matrix $X_{g}$ is not less than $c$. Suppose that when the base function $f$ solves $\left(\mathcal{G}\_{1},\mathcal{G}\_{2},\mathbf{z}^{1}+\varepsilon, \mathbf{z}^{2}\right)$, its output matrices whose similarity to $X_{g}$ is not less than $c$ are returned with probability $p$:
$$
\mathcal{X^{0}} = \{X_{i} \vert \frac{X_{i} \cdot X_{g} }{X_{g} \cdot X_{g}} \geq c, X_{i} \in \mathcal{X}\},\\
p = \mathbb{P}(X_{i} \vert X_{i} \in \mathcal{X^{0}})
$$
Our main result is that smoothed function $g$ is robust within a $\ell_{2}$ certified space, which also holds if we replace $p$ with a lower bound $\underline{p}$.

**Theorem** *Let $f$ be a matching function, $g$ be defined as in the equation above, and $\varepsilon \sim \mathcal{N}\left(0,\mathbf{\Sigma}\right)$. Suppose $X_{A} \in \mathcal{X^{0}}$ and $\underline{p}\in (\frac{1}{2},1]$ satisfy:
$$
% \mathbb{P}(X_{A} \vert X_{A} \in \mathcal{X^{0}}) \geq \underline{p}.
\mathbb{P}(f\left(\mathcal{G}\_{1},\mathcal{G}\_{2},\mathbf{z}^{1}+\varepsilon, \mathbf{z}^{2}\right)=X_{A}, X_{A} \in \mathcal{X^{0}})\geq \underline{p}.
$$
Then we can get the certified $\ell_{2}$ space for the addictive noise $\delta$:
$$
\|\|\delta^{\top} \mathbf{B}^{-1} \|\|< \Phi^{-1}\left(\underline{p}\right),
$$
which guarantees $g\left(\mathcal{G}\_{1},\mathcal{G}\_{2},\mathbf{z}^{1}+\delta, \mathbf{z}^{2}\right) \in \mathcal{X^{0}}$.  $\mathbf{B}^{\top} \mathbf{B}=\mathbf{\Sigma}$ and $\mathbf{B} \in \mathbb{R}^{n_{1} \times n_{1}}$ is a full rank and real symmetric matrix based on the node correlation in node matrix $\mathbf{z}^{1}$.*
Through this theorem we can get the certified robustness of $X$ as a whole. For specific details and related experimental results, please refer to Appendix E in the paper.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer 7S16 (1/4)
comment: >***Q1: The authors do not motivate the need for solving their problem but instead choose to start describing their approach in the introduction.***
>
**R1:** 
Thank you very much for your valuable suggestion.  We plan to add more related to motivations in the introduction.  In detail, our motivations are as follows:
As one of the common combinatorial optimization problems and most important research topics in the graph domain with wide applications in computer vision and pattern recognition, graph matching has been intensively studied over the decades, such as image keypoint matching [1], scene graph discovery [2] and vision-text retrieval [3].
* Studies on robustness of machine learning models have attracted wide attention, while the robustness of combinatorial solvers (graph matching is one the cases) is an emerging and unmatured topic whose importance has also been elaborated in the previous works [4]. 
* Graph matching (GM) problem is an intensively studied combinatorial optimization problem in computer vision and pattern recognition (see the surveys [15,16,17]). CVPR'22 paper [4] raised people's attention that neural GM solvers can be sensitive to (additive) noise perturbations on image pixel as well as in graph structure. More broadly speaking, some concurrent works on arxiv also try to formulate and address the attack and robustness of combinatorial solvers (both neural and learning-free ones) [5,6]. Yet these existing attack/defense works for graph or specifically combinatorial optimization on graphs are all using heuristic and cannot provide a theoretically guaranteed robustness. 
* Instead our approach can certify (Eq. 6 and 9) a GM solver's robustness radius within which the perturbation will not change the solution, regardless of its attack strategy which can be even unknown. Our certification can be applied to both learning-based and -free solvers. 
* Our certified algorithms are more suitable for applications with high security requirements to ensure consistent prediction results with minimal loss of accuracy. We hope our research can shed light on the mission-critical applications. For example, the use of cross-modal graph matching can assist in the decision and analysis of autonomous driving. Combining with our certified robustness method can improve the safety of driving in reality. 
* In addition, our approach is promising to be extended to other combinatorial optimization solvers soon, forming a general framework that always guarantees robustness. We have updated Introduction section in the paper.

>***Q2: Randomized smoothing makes inference slow due to the need for sample multiple points and also makes its prediction non-deterministic. Why is this a good approach for proving certified robustness for the GM setting?***
>
**R2:** Thank you for your valuable questions and we fully agree with you about the drawbacks of randomized smoothing. The main reasons why introducing the randomized smoothing approach to address the certified robustness of GM are as follows:
* First, randomized smoothing is feasible on large networks, therefore it is suitable for deep graph matching networks. Moreover, it makes no assumption about the base function’s architecture, thus a general certified robustness framework for many types of graph matching functions can be constructed, which also facilitates subsequent extensions to more combinatorial optimization problems.
Other common methods to certify robustness include: Lipschitz constant method [11,12], Interval Bound Propagation (IBP) methods [13,14], etc. These two methods are relatively flexible and don not require inference by sampling, but they have requirements on the network structure and the robust boundary obtained by the calculation is relatively loose. More importantly, they are not suitable for use in deep GM algorithms and Pascal VOC dataset.
* Second, we find that the smoothing distribution in randomized smoothing can be used to represent the structural correlation in the graph matching input which is beneficial to improve the overall certified robustness.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer 7S16 (2/4)
comment: >***Q3: The authors do not describe the impact of certified robustness on the accuracy of the classifier. Is there a tradeoff here? For vision models, humans are the oracle and the goal is to achieve the same level of robustness as them. I do not see such an oracle for GM as even non-learned models are not robust. What level of accuracy and robustness is good in this setting?***
>
**R3:** Thank you for your valuable comments. 
* First, there is indeed tradeoff between the certified robustness and the accuracy. In this paper, we use certified radius $\|\|\delta\|\|$ in Eq.9,12,13 to represent certified robustness which means the matching result remains unchanged if perturbing the node within $\|\|\delta\|\|$. Fig.2 shows that usually the larger the robust radius, the lower the certified accuracy. To compare the relationship more quantitatively, we use a table to reveal the tradeoff between the certified robustness and the accuracy. We show the standard accuracy and certified accuracy of different certified radius $\|\|\delta\|\|_{\text{lower}}$ of four methods (when $\sigma$ = 5) on NGMv2 GM algorithm and on Pascal VOC dataset.

|  | STD.ACC(%) |CERT.ACC (%)R=3.5 |CERT.ACC (%)R=7.0 |CERT.ACC (%)R=10.5 |CERT.ACC (%)R=14.0 |
| --- |  --- | --- | --- | --- | --- |
RS| 76.7 |**66.9**|0.0 |0.0 |0.0 |
DDRS| **77.4** |66.6 |50.5 |18.2 |0.0 |
 ANCER|76.5 |64.2 |49.1 |23.8 |7.8 |
  SCR-GM| 75.6 |63.7 |**51.5** |**36.4**|**12.2**|

* Second, $\mathbf{z}^1$ and $\mathbf{z}^{2}$ in the graph matching problem represent the location of the nodes. In the dataset Pascal VOC, it is annotated by human labeling or keypoint detectors. Humans can easily complete the matching tasks, therefore, humans are also the oracle for visual GM problem.
* Third, to evaluate the entire performance of the method, we follow [7] to use Average Certified Radius (ACR) proposed by [7] as follow:
$$
\mathbb{E}\_{x, y}\left[R\mathbb{1}(g(x)=y)\right].
$$
In our method, $g$ represents the smoothed function defined in Eq.4, $x$ represents the input node in the test dataset, and $y$ represents its ground truth matching node. $R$ represents the certified radius calculated by Eq.9, Eq.12, Eq.13, $\mathbb{1}$ is an indicator function.
We calculate the ACR $\|\|\delta\|\|_{\text{lower}}$ of four different RS-type methods ($\sigma$ = 5) and four GM methods.

|        | NGMv2     | CIE-H     | PCA-GM    | GMN       |
| ------ | --------- | --------- | --------- | --------- |
| RS     | 4.189     | 2.880     | 2.745     | 2.037     |
| DDRS   | 5.936     | 3.505     | 3.307     | 2.741 |
| ANCER  | 6.300     | 3.367     | 3.179 | 2.517     |
| SCR-GM | **7.107** | **3.726** | **3.455**     |**2.745**     |

We also made a modification in Section 5.1 and 5.2.

>***Q4: Randomized smoothing for neural networks was introduced by https://arxiv.org/abs/1802.03471 and not Cohen et al. The authors do not cite this.***
>
**R4:** Thank you for your suggestion and we agree with your comment. (Lecuyer et al.2019) [8] proposed randomized smoothing firstly as a certified adversarial defense, and used it to train the first certifiably robust classifier for ImageNet. However, its guarantees are loose, (Cohen et al, 2019) [9] solved this problem and proposed the tight robustness guarantee for randomized smoothing. Therefore, our method takes (Cohen et al, 2019) [9] as the baseline. We will revise the article and add citation to [8]. We also made a modification in Section 2.1.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer 7S16 (3/4)
comment: >***Q5: $\Sigma$ is an important hyperparameter controlling the robustness/accuracy tradeoff but the authors do not elaborate on the principles behind its construction (e.g., what properties it should satisfy).  The authors should consider performing a study on the impact of different choices for constructing $\Sigma$ on the certified robustness.***
>
**R5:** Thank you for your valuable questions. $\mathbf{\Sigma}$ should be a positive definite matrix in the paper which enables us to derive certified space using linear transformations in Appendix. Therefore, we construct $\mathbf{\Sigma}$ by constructing $\mathbf{B}$ using $\mathbf{B}^{\top} \mathbf{B}=\mathbf{\Sigma}$, thus we can maintain the properties of $\mathbf{\Sigma}$.

In our work, we use Eq.10 to construct $\mathbf{B}$ where $d_{mn}$ is an euclidean distance in our work, which is a method to reveal the correlation between nodes. And we evaluate the effect of the correlations strength in $\mathbf{\Sigma}$ on robustness in Fig.4(a). We compare the experimental results of retaining correlations, partially retaining correlations (for example, 20% means that the correlation coefficients of the top 20% are retained), and completely removing correlations to demonstrate the effectiveness of the $\mathbf{\Sigma}$. The correlation enhancement in $\mathbf{\Sigma}$ improves the robust performance overall.

Moreover, we fully agree with your proposal to performing a study on the impact of different choices for constructing $\mathbf{\Sigma}$. Therefore, we use cosine similarity, pearson similarity, dice similarity to construct $\mathbf{B}$ respectively and then verify the effect of the different $\Sigma$ obtained on the robustness results. We introduce the similarity as follows:
For two points in the Euclidean space $\mathbb{R}^n$: $A=\left(a_1, a_2, \cdots, a_n\right)$ and $B=\left(b_1, b_2, \cdots, b_n\right)$, where $A$ and $B$ cannot be zero point at the same time:
* $\operatorname{Cosine} \operatorname{Similarity}(A, B)=\frac{A \cdot B}{\|A\|\_2 \cdot\|B\|\_2}=\frac{\sum_{i=1}^n a_i b_i}{\sqrt{\sum_{i=1}^n a_i^2} \cdot \sqrt{\sum_{i=1}^n b_i^2}} \in[-1,1].$
* $\operatorname{Pearson} \operatorname{Similarity}(A, B)=\frac{\operatorname{cov}(A, B)}{\sigma_A \cdot \sigma_B}=\frac{\sum_{i=1}\left(a_i-\bar{A}\right) \cdot\left(b_i-\bar{B}\right)}{\sqrt{\sum_{i=1}^n\left(a_i-\bar{A}\right)^2} \cdot \sqrt{\sum_{i=1}^n\left(b_i-\bar{B}\right)^2}} \in [-1,1],$
where $\bar{A}=\sum_{i=1}^n a_i / n, \bar{B}=\sum_{i=1}^n b_i / n$.
* $\operatorname{Dice} \operatorname{Similarity}(A, B)=\frac{2 \sum_{i=1}^n a_i b_i}{\sum_{i=1}^n\left(a_i^2+b_i^2\right)} \in [-1,1]$.

We compare the certified robustness using four methods to construct matrix $\mathbf{B}$ on NGMv2 GM algorithm and on Pascal VOC dataset.


|  | STD.ACC (%) |CERT.ACC (%)R=3.0 |CERT.ACC (%)R=6.0 |CERT.ACC (%)R=9.0 |CERT.ACC (%)R=12.0 |CERT.ACC (%)R=15.0 |
| --- |  --- | --- | --- | --- | --- | --- |
Euclidean| 75.2 |64.3 |**53.3** |**42.0** |**24.1** |**6.7** |
Dice| **75.6** |**65.5** |52.3 |41.0 |23.5 |6.6 |
Cosine| 75.6 |65.1 |52.0 |41.0 |23.5 |6.6 |
Pearson| 75.6 |65.2 |51.8 |40.7 |23.6 |6.6 |

We also made a modification in Sec 4.3 and Appendix C.4 and D.1.

>***Q6: In the equation for calculating $p_{B}$, the authors use $s_{i}\neq p_{A}$ which is semantically not correct.***
>
**R6:** Thank you for your valuable suggestion. We modify the defination as: Suppose that when the base function $f_{i}$ solves for the optimal matching of node $i$ in $\mathbf{z}^{1}$, the most probable node $r_{A}$ in $\mathbf{z}^{2}$ is returned with probability $p_{A}=\max_{s_{i} \in \mathbf{S}\_{i}}s_{i}$, where $\mathbf{S}\_{i}$ is the $i$-th row of $\mathbf{S}$. Similarly, the probability of ''runner-up'' node $r_{B}$ in $\mathbf{z}^{2}$ is denoted as $p_{B}$, $p_{B}=\max_{s_{i} \in \mathbf{S}\_{i}, r_{B}\neq r_{A}}s_{i}$. 
We also made a modification in Section 4.1.

>***Q7: In figure 2, how are the y-axis values computed? are they averaged over the whole dataset?***
>
**R7:** Thank you for your valuable question. The certified accuracy indicates the percentage of nodes that match accurately among the nodes whose matching radius is not less than the radius ($x$-axis), so it is not averaged over the whole dataset. In the experiment, for each input in the test set, our method outputs the corresponding robustness radius $\|\|\delta\|\|$ of each node in the input and whether the match correctly. Then the certified accuracy is defined as:
$$
CA(R)=\mathbb{E}_{x, y}\left[\mathbb{1} (\|\|\delta\|\| \geq R)\mathbb{1}(g(x)=y)\right].
$$
In our method, $g$ represents the smoothed function defined in Eq.4, $x$ represents the input node in the test dataset, and $y$ represents its ground truth matching node. $\|\|\delta\|\|$ represents the certified radius calculated by Eq.9, Eq.12, Eq.13, $R$ is the scale of the $x$-axis, $\mathbb{1}$ is an indicator function. We also made a modification in page 5.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer 7S16 (4/4)
comment: >***Q8: Would the radii be larger if one considered the robustness of three nodes at a time instead of a single node as considered here? In general, considering the robustness of the full matrix should reveal more correlations right? which should yield larger radii. Did you choose single-node decompositions because it makes the problem tractable?***
>
**R8:** This is a very valuable and constructive question.

*  In this paper, we focus on the certified robustness of node matching results in the graph rather than the whole graph matching result in evaluation. Our work treats the GM solver as blackbox to get the relaxed matching $\mathbf{S}$, then uses a post-binarization step to modify the output format $\mathbf{X}$ and get the node matching function set $\mathbf{F}$. Then we certify the robustness of $\mathbf{F}$.
* As you said, considering the robustness of the full matrix is also valuable and promising research work which is able to utilize more graph structure information. We also propose another method of certifying the robustness of the entire matrix $X$. Please refer to general response and Appendix.E for details.

>***Q9: The authors do not discuss the impact of sampling on the inference speed.***
>
**R9:** Thank you for your suggestion. In fact, Monte Carlo sampling reduces the speed of our inference, which is the main drawback of introducing the RS algorithm to our method. However, as written in R2, due to the flexibility of the RS algorithm and its properties suitable for graph matching, we introduce it to solve the certified robustness problem. In the future, we can refer to the idea of [10] to improve the inference speed of the algorithm.

[1] R. Wang, J. Yan, and X. Yang. Learning combinatorial embedding networks for deep graph matching. In Int. Conf.
Comput. Vis., pages 3056–3065, 2019. 1, 2, 3, 7.

[2] Lichang Chen, Guosheng Lin, Shijie Wang, and Qingyao Wu. Graph edit distance reward: Learning to edit scene graph. Eur. Conf. Comput. Vis., 2020. 1

[3] Yu Xiong, Qingqiu Huang, Lingfeng Guo, Hang Zhou, Bolei Zhou, and Dahua Lin. A graph-based framework to bridge movies and synopses. In Int. Conf. Comput. Vis., October 2019. 1

[4] Ren Q, Bao Q, Wang R, et al. Appearance and Structure Aware Robust Deep Visual Graph Matching: Attack, Defense and Beyond[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 15263-15272.

[5] Lu et al., A General Framework for Evaluating Robustness of Combinatorial Optimization Solvers on Graphs, arXiv:2201.00402

[6] Geisler et al., Generalization of Neural Combinatorial Solvers Through the Lens of Adversarial Robustness, arXiv:2110.10942

[7] Zhai R, Dan C, He D, et al. MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius[C]//International Conference on Learning Representations. 2019.

[8] Lecuyer M, Atlidakis V, Geambasu R, et al. Certified robustness to adversarial examples with differential privacy[C]//2019 IEEE Symposium on Security and Privacy (SP). IEEE, 2019: 656-672.

[9] Cohen J, Rosenfeld E, Kolter Z. Certified adversarial robustness via randomized smoothing[C]//International Conference on Machine Learning. PMLR, 2019: 1310-1320.

[10] Chen R, Li J, Yan J, et al. Input-specific robustness certification for randomized smoothing[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(6): 6295-6303.

[11] Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks. In Conference on Neural Information Processing Systems (NeurIPS), 2018.

[12] Cem Anil, James Lucas, and Roger B. Grosse. Sorting out Lipschitz function approximation. In International Conference on Machine Learing (ICML), 2019.

[13] Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. "Efficient neural network robustness certification with general activation functions." NeurIPS 2018. 

[14] Wang S, Zhang H, Xu K, et al. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network 

robustness verification[J]. Advances in Neural Information Processing Systems, 2021, 34: 29909-29921.

[15] Fifty years of graph matching, network alignment and network comparison, Information Sciences 2016

[16] A short survey of recent advances in graph matching, ICMR 2016

[17] Learning for Graph Matching and Related Combinatorial Optimization Problems, IJCAI 2020

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer BhnR (1/2)
comment: >***Q1: It's not obvious that there will ever be any application of these ideas (quite possibly low significance).***
> 
**R1:** Studies on robustness of machine learning models have attracted wide attention, while the robustness of combinatorial solvers (graph matching is one the cases) is an emerging and unmatured topic whose importance has also been elaborated in the previous works [1].

* Graph matching (GM) problem is an intensively studied combinatorial optimization problem in computer vision and pattern recognition (see the surveys [6,7,8]). CVPR'22 paper [1] raised people's attention that neural GM solvers can be sensitive to (additive) noise perturbations on image pixel as well as in graph structure. More broadly speaking, some concurrent works on arxiv also try to formulate and address the attack and robustness of combinatorial solvers (both neural and learning-free ones) [4,5]. Yet these existing attack/defense works for graph or specifically combinatorial optimization on graphs are all using heuristic and cannot provide a theoretically guaranteed robustness. 
* Instead our approach can certify (Eq. 6 and 9) a GM solver's robustness radius within which the perturbation will not change the solution, regardless of its attack strategy which can be even unknown. Our certification can be applied to both learning-based and -free solvers. 
* Our certified algorithms are more suitable for applications with high security requirements to ensure consistent prediction results with minimal loss of accuracy. We hope our research can shed light on the mission-critical applications. For example, the use of cross-modal graph matching can assist in the decision and analysis of autonomous driving. Combining with our certified robustness method can improve the safety of driving in reality. 
* In addition, our approach is promising to be extended to other combinatorial optimization solvers soon, forming a general framework that always guarantees robustness. We have updated Introduction section in the paper.

>***Q2: Information provided on experiments makes it hard to quantify the improvement. Need metrics like AUC to quantize the performance better. (is AUC even the right metric?)  It's very hard to quantify with the provided results (basically guessing about one line vs another)***
>
**R2:** Thank you for your valuable suggestion. 
To evaluate our method, we mainly follow evaluation criteria in [2] and use a plot to visualize the relationship between the robustness radius and the certified accuracy as in Fig.2. Although Fig. 2 allows us to visually compare the methods, a quantitative evaluation method is also needed.

Therefore, we totally agree with you that we need a metric to show the superiours of our method on the certified robustness over the whole dataset. Thus, we introduce Average Certified Radius (ACR) proposed by [3] to our methods as follows:
$$
\mathbb{E}_{x, y}\left[R\mathbb{1}(g(x)=y)\right].
$$
In our method, $g$ represents the smoothed function defined in Eq.4, $x$ represents the input node in the test dataset, and $y$ represents its ground truth matching node. $R$ represents the certified radius calculated by Eq.9, Eq.12, and Eq.13; meanwhile, $\mathbb{1}$ is an indicator function.

|        | NGMv2     | CIE-H     | PCA-GM    | GMN       |
| ------ | --------- | --------- | --------- | --------- |
| RS     | 4.189     | 2.880     | 2.745     | 2.037     |
| DDRS   | 5.936     | 3.505     | 3.307     | 2.741 |
| ANCER  | 6.300     | 3.367     | 3.179 | 2.517     |
| SCR-GM | **7.107** | **3.726** | **3.455**     |**2.745**     |

We calculate the ACR with $\|\|\delta\|\|_{\text{lower}}$ of four different RS-type methods ($\sigma$ = 5) and four GM methods which indicates that our method shows a better certified robustness performance over the test set. We have had a synchronous modification in Section 5.1 and Appendix D.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer BhnR (2/2)
comment: >***Q3: Authors don’t have a good explanation for interpreting results either.***
**R3:** Thank you for your valuable questions, and we further interpret the experimental results as follows.
* In Section 5.2, we evaluate the certified robustness on deep networks when perturbing node locations. We also evaluate the certified robustness on deep networks when perturbing features in Appendix B.
We follow the evaluation metrics of [2] to compare our method and baselines by robustness radius $R$ ($\|\|\delta\|\|\_{\text{lower}}$, $\|\|\delta\|\|\_{\text{max}}$ and $\|\|\delta\|\|\_{\text{volume}}$ in paper) and certified accuracy. When the radius on $x$-axis is the same, the higher the certified accuracy on $y$-axis, the better the certified robustness of the method. For a single sample, we output the matching result of smoothed $g$ (Eq.4) and robustness radius $R$. We aggregate the results of all samples into a plot with $R$ as the $x$-axis and accuracy as the $y$-axis as in Fig.2. 
The certified accuracy of our method is slightly lower sometimes than baselines when $\|\|\delta\|\|\_{\text{lower}}$ is small. However, when $\|\|\delta\|\|\_{\text{lower}}$ is large, the accuracy of baseline decreases significantly or even fails completely while our method maintains a more respectable accuracy. When evaluating using $\|\|\delta\|\|\_{\text{max}}$ and $\|\|\delta\|\|\_{\text{volume}}$, the advantages of our method are even more obvious. 
In conclusion, our method outperforms other methods in certified robustness, and the advantage is especially evident when the perturbation is large. 
* In Section 5.3, we evaluation the certified robustness of non-learning solvers RRWM on simulation dataset, we also follow the evaluation metrics of [2]. We compare the effectiveness of our method with the baseline by perturbing node locations and edge features respectively. Our method has similar performance of the certified accuracy corresponding to the same $\|\|\delta\|\|\_{\text{lower}}$ with the baseline. However, our method performs better on $\|\|\delta\|\|\_{\text{volume}}$ and $\|\|\delta\|\|\_{\text{max}}$ which indicates that the guarantee space certified by our method is wider and its overall robustness is better.
* In Section 5.4, we compare the effect of the correlations strength in $\Sigma$ on robustness in Fig. 4(a). We compare the experimental results of retaining correlations, partially retaining correlations (for example, 20% means that the correlation coefficients of the top 20% are retained), and completely removing correlations to demonstrate the effectiveness of the $\Sigma$. Then we find that $\Sigma$ can trade off robustness radius and certified accuracy as in Fig. 4(b). When the $\Sigma$ is calculated by larger original $\sigma$, the radius range is wider but some accuracy is sacrificed. This experiment reveals how to set the parameters according to the requirements for application.
* We made a synchronous modification in the Section.5.


[1] Ren Q, Bao Q, Wang R, et al. Appearance and Structure Aware Robust Deep Visual Graph Matching: Attack, Defense and Beyond[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 15263-15272.

[2] Cohen J, Rosenfeld E, Kolter Z. Certified adversarial robustness via randomized smoothing[C]//International Conference on Machine Learning. PMLR, 2019: 1310-1320.

[3] Zhai R, Dan C, He D, et al. MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius[C]//International Conference on Learning Representations. 2019.

[4] Lu H, Li Z, Wang R, et al. Mind Your Solver! On Adversarial Attack and Defense for Combinatorial Optimization[J]. arXiv preprint arXiv:2201.00402, 2021.

[5] Geisler et al., Generalization of Neural Combinatorial Solvers Through the Lens of Adversarial Robustness, arXiv:2110.10942

[6] Fifty years of graph matching, network alignment and network comparison, Information Sciences 2016

[7] A short survey of recent advances in graph matching, ICMR 2016

[8] Learning for Graph Matching and Related Combinatorial Optimization Problems, IJCAI 2020

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer n9K9 (1/2)
comment: >***Q1: In section 4.1 the point attributes become 2-dimensional whereas the original formulation was with an arbitrary dimension d, and whether this matters for the results in the paper.***
>
**R1:** Thank you for your careful reading and meaningful questions. The input of graph matching algorithms is $(\mathcal{G}\_{1},\mathcal{G}\_{2},\mathbf{z}^{1},\mathbf{z}^{2})$, where $\mathcal{G}\_{1}$ and $\mathcal{G}\_{2}$ are two input graphs, $\mathbf{z}^{1}\in\mathbb{R}^{n_1\times 2}$ and $\mathbf{z}^{2}\in\mathbb{R}^{n_2\times 2}$ are annotated nodes of graphs. $\mathcal{G}\_{1}=(\{\mathbf{V}\_{1}\in\mathbb{R}^{d_v\times n_1}, \mathbf{E}\_{1}\in\mathbb{R}^{d_e\times m_1}\})$, $\mathbf{V}\_{1}$ and $\mathbf{E}\_{1}$ represent the feature matrix of $n_{1}$ nodes and $m_{1}$ edges (likewise for $\mathbf{V}\_{2}$ and $\mathbf{E}\_{2}$). 

In most our discussion, we focus on the certified robustness research by perturbing the position attribute of nodes $\mathbf{z}^{1}$. In Section 4.1, $\mathbf{z}^{1}\in\mathbb{R}^{n_1\times 2}$ reperensents the node coordinates which are usually annotated by human labeling or node detectors. The location of nodes suffers from the labeling randomness, so its robustness is very valuable for research.

Moreover, in Appendix B, we extend the certified robustness work from perturbating node position to node features, which is $d_v$-dimension. In section4.1, we derive Theorem1 by perturbation $\mathbf{z}^{1}$ as an example, but in fact this conclusion also applies to perturbation features, as in the Appendix B. We also made a modification in the Introduction section.

>***Q2: In the definition of F (same section) it was not clear what is the range of the $f_i$'s (meaning that $r_j$ is a node in G2) until discerning this indirectly from the later derivations.***
>
**R2:** Thank you for your very valuable suggestion, and we will add an explanation for the composition of $\mathbf{F}$: $\mathbf{F}=\{f_{i}\vert f_{i}:\left(\mathcal{G}\_{1},\mathcal{G}\_{2},\mathbf{z}^{1}, \mathbf{z}^{2}\right)\rightarrow r_{j}, i\in n_{1}, j \in n_{2}\}$, $f_{i}$ solves for which node in $\mathbf{z}^{2}$ matches the i-th node in $\mathbf{z}^{1}$, $r_{j}$ represents the j-th node $r_{j}$ in $\mathbf{z}^{2}$.
We have added this explanation of $\mathbf{F}$ in Section 4.1.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer n9K9 (2/2)
comment: >***Q3: I am not sure what I'm supposed to be seeing in figure 1(b), which looks like four near-identical figures.***
>
**R3:** Thank you for your very valuable question. In the original Fig.1(b), the colored hot regions represent the Monte Carlo samples which are around the nodes in $\mathbf{z^{1}}$ using the four distributions introduced in Fig.1(a). 

We recognize that Fig.1(b) is not clear to represent our original idea on showing the difference of the certified radius, we modify the figure by drawing the certified radius $\|\|\delta\|\|\_{\text{lower}}$ of the four methods instead of the sampling results of the four smoothing distributions. The center of the circle in the new Fig.1(b) represents the position of the node in $\mathbf{z^{1}}$, while the radius of the circle represents its corresponding certified radius $\|\|\delta\|\|\_{\text{lower}}$ calculated by Eq.9. The new Fig.1(b) shows that the certified radius obtained by our method is larger overall, so our method has superior performance. 

We further calculate the average certified radius (ACR) of the four methods for nodes in this picture to evaluate quantitatively. The Average Certified Radius (ACR) is proposed by [1] as follows:
$$
\mathbb{E}_{x, y}\left[R\mathbb{1}(g(x)=y)\right].
$$
In our method, $g$ represents the smoothed function defined in Eq.4, $x$ represents the input node in the test dataset, and $y$ represents its ground truth matching node. $R$ represents the certified radius calculated by Eq.9, Eq.12, Eq.13, $\mathbb{1}$ is an indicator function.

We calculate the ACR $\|\|\delta\|\|\_{\text{lower}}$ for this sample in Fig.2(b) using four different RS-type methods and NGMv2 GM algorithm. The ACR results of the four certification methods are $ACR=\{5.13,6.75,7.75,8.53\}$. We also made a modification in page 5.

>***Q4: And finally, I was not sure if and how your method makes sure that the robust matching mechanism maintains the marginal constraints of the relaxed matching (the last two constraints in equation (1)).***
>
**R4:** This is a very valuable and constructive question. 
* In this paper, we focus on the certified robustness of node matching results in the graph rather than the whole graph matching result in evaluation. Our work treats the GM solver as blackbox to get the relaxed matching $\mathbf{S}$, then uses a post-binarization step to modify the output format $\mathbf{X}$ and get the node matching function set $\mathbf{F}$. We certify the robustness of $\mathbf{F}$, so it is not necessary to keep the constraint of $\mathbf{X}$ all the time. 
* We summarize the relationship between our certified results and $\mathbf{X}$ constraints. The constraints in Eq.1 may not be always satisfied, while the constraints must be satisfied when all nodes in the graph are matched correctly meaning that the entire graph is correctly matched.
* As you said, certifying the robustness of graph matching without breaking the constraints is very valuable and promising research work which is able to utilize more graph structure information.
We propose another method of certifying the robustness of the entire matrix $X$, which always maintains the constraints of $X$. Please refer to Appendix.E and General Response for details.


[1] Zhai R, Dan C, He D, et al. MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius[C]//International Conference on Learning Representations. 2019.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer Sgyn (2/2)
comment: >***Q2: Figure 1(b) is hard for me to detect differences between methods.***
>
**R2:** Thank you for your very valuable question. In the original Fig.1(b), the colored hot regions represent the Monte Carlo samples which are around the nodes in $\mathbf{z^{1}}$ using the four distributions introduced in Fig.1(a). 

We recognize that Fig.1(b) is not clear to represent our original idea on showing the difference of the certified radius, we modify the figure by drawing the certified radius $\|\|\delta\|\|\_{\text{lower}}$ of the four methods instead of the sampling results of the four smoothing distributions. The center of the circle in the new Fig.1(b) represents the position of the node in $\mathbf{z^{1}}$, while the radius of the circle represents its corresponding certified radius $\|\|\delta\|\|\_{\text{lower}}$ calculated by Eq.9. The new Fig.1(b) shows that the certified radius obtained by our method is larger overall, so our method has superior performance. 

We further calculate the average certified radius (ACR) of the four methods for nodes in this picture to evaluate quantitatively. The Average Certified Radius (ACR) is proposed by [5] as follows:
$$
\mathbb{E}_{x, y}\left[R\mathbb{1}\(g(x)=y)\right].
$$
In our method, $g$ represents the smoothed function defined in Eq.4, $x$ represents the input node in the test dataset, and $y$ represents its ground truth matching node. $R$ represents the certified radius, $\mathbb{1}\{\}$ is an indicator function. 

We calculate the ACR for this sample in Fig.2(b) using four different RS-type methods and NGMv2 GM algorithm when setting $R=\|\|\delta\|\|\_{\text{lower}}$. The ACR results of the four certification methods are $ACR=\{5.13,6.75,7.75,8.53\}$. We also made a modification in page 5.

[1] Cohen J, Rosenfeld E, Kolter Z. Certified adversarial robustness via randomized smoothing[C]//International Conference on Machine Learning. PMLR, 2019: 1310-1320.

[2] Gao Z, Hu R, Gong Y. Certified robustness of graph classification against topology attack with randomized smoothing[C]//GLOBECOM 2020-2020 IEEE Global Communications Conference. IEEE, 2020: 1-6.

[3] Levine A, Feizi S. (De) Randomized smoothing for certifiable defense against patch attacks[J]. Advances in Neural Information Processing Systems, 2020, 33: 6465-6475.

[4] Pérez J C, Alfarra M, Giancola S, et al. 3DeformRS: Certifying Spatial Deformations on Point Clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 15169-15179.

[5] Zhai R, Dan C, He D, et al. MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius[C]//International Conference on Learning Representations. 2019.

From: ICLR.cc/2023/Conference/Paper3619/Authors
title: Response to Reviewer Sgyn (1/2)
comment: >***Q1: The techniques used to derive the theoretical results seem quite similar to the original randomized smoothing method (Cohen et al, 2019). The main difference is the noisy distribution here depends on the correlation between nodes, which is solved with standard linear algebra analysis. My only concern is that the techniques used to derive the theoretical results seem quite similar to the original randomized smoothing method (Cohen et al, 2019).***
>
**R1:** Thank you for your valuable questions, which we will answer in two aspects.

From one aspect, the method proposed by [1] is a fundamental theory in the field of certified robustness, and this method only addresses the classification problem. Then many certified robustness studies in other fields are based on this theory and have similar expressions on the robustness space as Eq.3 (e.g., certified robustness for graph classification [2], Patch attack [3], Point Clouds [4]). However, in fact the treatment of input distribution, perturbation space, output form and the meaning of the parameters in these methods is relevant and innovative，our method is also similar. 

From another aspect, our contribution and differentiate point from [1] are as follows:
* First, we present the first definition (Eq. 5) of certified robustness for graph matching by introducing the RS certification analysis to the node matching stage. The research of the certified robustness of GM is an emerging and unmatured topic, which can improve the security and usability of GM algorithms.
* Second, we propose to construct a smoothing distribution using $\Sigma$ containing node correlations (for example, in Euclidean space nodes with closer distance are more similar), and use the transformation property of the positive semi-definite real symmetric matrix in Appendix A to obtain a certified space in Eq.6 which is a more general representation (rather than a radius). 
* Third, $\underline{p_{A}}$ and $\overline{p_{B}}$ in Eq.6 represent the matching probability rather than classification probability in [1]. For comparison with baselines, we use the eigenvalue property in Lemma1 to further obtain a certified radius ( $\|\|\delta\|\|\_{\text{lower}}$ ) and two auxiliary evaluation radii ($\|\|\delta\|\|\_{\text{max}}$ and $\|\|\delta\|\|\_{\text{volume}}$).
* Fourth, to certify the robustness of graph matching, we process and modify its output $\mathbf{X}$. In the paper, we use a post-binarization step that converts the relaxed matching $\mathbf{S}$ (by an arbitrary relaxed GM solver) to the node matching function set $\mathbf{F}$. Then we can certify the robustness of $\mathbf{F}$.
* In summary, we deal with the input and output properties of the graph matching problem and propose a structural smoothing distribution to construct a certified robustness function. Our method presents the first definition and strategy of certified robustness for graph matching and outperforms baselines in experiments.

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_7S16
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper considers the problem of the probabilistic robustness of graph-matching (GM) algorithms against norm-based adversarial perturbations. The authors develop new algorithms for randomized smoothing for GM by exploiting the structure of the problem. They start by decomposing the robust matching problem defined over the whole graph into subproblems defined over the individual nodes. Next, the noise for building the smoothed classifier is sampled from a joint gaussian distribution which is set up in a way to capture correlations between the different nodes in the graph. Experimental evaluation is performed on the popular Pascal-VOC dataset showing the effectiveness of the proposed method.
strength_and_weaknesses: Strengths:
1. The technical contributions for building a probabilistically robust classifier for GM presented here are novel and solid. I wonder if structured distributions can also be used for improving the certified radius of vision classifiers.

2. The experimental results show that the proposed method can certify larger radii than standard baselines.

Weaknesses:
1. The problem and the approach are not well-motivated. While the robustness of vision models is necessary when deployed in safety-critical scenarios such as self-driving cars, it is not clear what security implications the non-robustness of graph matching can cause. The authors do not motivate the need for solving their problem but instead choose to start describing their approach in the introduction. 

2. Randomized smoothing makes inference slow due to the need for sample multiple points and also makes its prediction non-deterministic. Why is this a good approach for proving certified robustness for the GM setting?

3. The authors do not describe the impact of certified robustness on the accuracy of the classifier. Is there a tradeoff here?  For vision models, humans are the oracle and the goal is to achieve the same level of robustness as them. I do not see such an oracle for GM as even non-learned models are not robust. What level of accuracy and robustness is good in this setting?
clarity,_quality,_novelty_and_reproducibility: Clarity and quality: Besides the issues with clarity mentioned above, I have the following comments for the authors:

1. Randomized smoothing for neural networks was introduced by https://arxiv.org/abs/1802.03471 and not Cohen et al. The authors do not cite this. 

2.$\sum$ is an important hyperparameter controlling the robustness/accuracy tradeoff but the authors do not elaborate on the principles behind its construction (e.g., what properties it should satisfy). Eq. (10) seems a bit random to me, why would any other equation not work? The authors should consider performing a study on the impact of different choices for constructing $\sum$ on the certified robustness. 

3. In the equation for calculating $p_B$, the authors use $s_i \neq p_A$ which is semantically not correct. 

4. In figure 2, how are the y-axis values computed? are they averaged over the whole dataset? 

5. Would the radii be larger if one considered the robustness of three nodes at a time instead of a single node as considered here? In general, considering the robustness of the full matrix should reveal more correlations right? which should yield larger radii. Did you choose single-node decompositions because it makes the problem tractable?

6. The authors do not discuss the impact of sampling on the inference speed.

Novelty: The technical ideas are sufficiently novel.

Reproducibility: Code is provided to ensure reproducibility, although I did not try to run it!
summary_of_the_review: In summary, this is a technically solid and novel work for developing smoothed graph-matching classifiers. However, I am not sure about the motivation for solving this problem and why randomized smoothing is a good approach for building robustness here.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_BhnR
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper provides a new result on certified robustness for graph matching.  Its based on randomized smoothing (Cohen 2019), but the authors use a correlation matrix based on the graph information to construct a joint Gaussian distribution for smoothing (vs the standard single Gaussian distribution).  The experimental results seem to show some effect, but it's very hard to quantify with the provided results (basically guessing about one line vs another).  This heavily detracts from the work.
strength_and_weaknesses: Strengths:
+ New results on the certified robustness for graph matching.
+ Experimental results seem to show a benefit 
+ Generally good write up, but weak experimental analysis

Weaknesses:
- It's not obvious that there will ever be any application of these ideas (quite possibly low significance)
- Information provided on experiments makes it hard to quantify the improvement.  Need metrics like AUC to quantize the performance better.  (is AUC even the right metric?)
- Authors don’t have a good explanation for interpreting results either

clarity,_quality,_novelty_and_reproducibility: Clarity: Fairly clear for the material as far as the theory goes.  Experimental results need more.

Quality: Generally high quality.

Novelty:  Harder to judge, but I think there's some novelty.  Significance might be low though.

Reproducibility: Code provided.
summary_of_the_review: This work might have enough to get in with better analysis of the existing experimental results.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_n9K9
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper studies the problem of certifying the robustness of structural graph matching, where the goal is to return a matching (or assignment) between the nodes of two input graphs G1 and G2, based on node and edge attributes. A robust graph matching mechanism is supposed to be resilient to adversarial perturbations of the input. The approach taken in this paper is to cast the graph matching problem as a collection of classification problems, where each node in G1 corresponds to a classification problem, and the nodes of G2 are the possible classes. Thus, if v_i is a node in G1 and u_j is a node in G2, then classifying v_i into the class u_j means that the matching matches v_i to u_j.

This formulation allows the paper to apply the generic randomizes smoothing (RS) technique to graph matching. RS is a general technique for turning non-robust classification mechanisms into certifiably robust ones, by adding some perturbation noise to the point that needs to classified, and classifying it to the most likely class (according to the input non-robust mechanism) of the perturbed point. Casting graph matching as a collection of classification problems as above allows applying RS, by adding noise to the nodes in G1. Furthermore, the covariance matrix of the Gaussian noise is chosen to reflect the structure of the input graphs, allowing for correlation between nodes, rather than adding independent noise, which according to the paper was the approach taken in prior work. 
strength_and_weaknesses: The paper appears to study a meaningful problem, and the proposed solution makes sense, up to some clarifications listed below. 
clarity,_quality,_novelty_and_reproducibility: The paper is generally well-written and I could largely understand and follow with almost no background on the subject, yet some parts were harder to follow. For example, I'm not sure why in section 4.1 the point attributes become 2-dimensional whereas the original formulation was with an arbitrary dimension d, and whether this matters for the results in the paper. In the definition of F (same section) it was not clear what is the range of the f_i's (meaning that r_j is a node in G2) until discerning this indirectly from the later derivations. I am not sure what I'm supposed to be seeing in figure 1(b), which looks like four near-identical figures. And finally, I was not sure if and how your method makes sure that the robust matching mechanism maintains the marginal constraints of the relaxed matching (the last two constraints in equation (1)).
summary_of_the_review: The paper ultimately makes sense to me in terms of the tackled problem and the proposed solution, without major issues that I could detect, and I think it can be accepted. I am, unfortunately, completely unfamiliar with almost all aspects of the topic of the paper (structural graph matching and certifiable robustness), so this should be taken as a non-expert review.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper3619/Reviewer_Sgyn
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper considers the certified robustness of graph matching. Graph matching aims to find a matching between nodes of two graphs that maximizes the overall affinity score. This paper defines the certified robustness on graph matching by using randomized smoothing to the node classification stage. Specifically, they consider the randomized smoothing for the mapping function of each node while using a joint smoothing distribution matrix. They show the strictly certifies radius and two extra radii for evaluation. Finally, their experimental results show their certified robustness method performs better than previous methods on real-world datasets. 
strength_and_weaknesses: Strengths:
1. This paper proposed a new setting for certified robustness of graph matching, which is a significant problem and can be used in many applications.
2. They provided solid theoretical results on the strictly certified radius.


Weaknesses:
1. The techniques used to derive the theoretical results seem quite similar to the original randomized smoothing method (Cohen et al, 2019). The main difference is the noisy distribution here depends on the correlation between nodes, which is solved with standard linear algebra analysis. 
2. Figure 1(b) is hard for me to detect differences between methods.
clarity,_quality,_novelty_and_reproducibility: This paper is well-written and easy to follow. The certified robustness on graph matching considered in this paper is a new setting. I think the results in this paper are original and reproducible. 
summary_of_the_review: Overall I think this paper is very interesting. It provides theoretical results and convincing experimental results. My only concern is that the techniques used to derive the theoretical results seem quite similar to the original randomized smoothing method (Cohen et al, 2019). 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold