Original: R EVOCABLE D EEP R EINFORCEMENT L EARNING WITH A FFINITY R EGULARIZATION FOR O UTLIER -R OBUST G RAPH M ATCHING
Revised: R EVOCABLE D EEP R EINFORCEMENT L EARNING WITH A FFINITY R EGULARIZATION FOR O UTLIER -R OBUST G RAPH M ATCHING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Graph matching (GM) has been a building block in various areas including computer vision and pattern recognition.
Revised: Graph matching (GM) has been a building block in various areas including computer vision and pattern recognition.

Original: Despitethe recent impressive progress, existing deep GM methods often have  difﬁculty in handling outliers, which are ubiquitous in practice.
Revised: Despite recent impressive progress, existing deep GM methods often have obvious difﬁculty in handling outliers, which are ubiquitous in practice.

Original: We propose a deep reinforcement learning based approach RGM, whose sequential node matching scheme naturally ﬁts the strategy for selective inlier matching against outliers.
Revised: We propose a deep reinforcement learning based approach RGM, whose sequential node matching scheme naturally ﬁts the strategy for selective inlier matching against outliers.

Original: A revocable action framework is devised to improve the agent’s ﬂexibility against the complex constrained GM task.
Revised: A revocable action framework is devised to improve the agent’s ﬂexibility against the complex constrained GM.

Original: Moreover, we propose a quadratic approximation technique to regularize the afﬁnity score, in the presence of outliers.
Revised: Moreover, we propose a quadratic approximation technique to regularize the afﬁnity score, in the presence of outliers.

Original: As such, the agent can ﬁnish inlier matching timely when the afﬁnity score stops growing, for which otherwise an additional parameter i.e. the number of inliers is needed to avoid matching outliers.
Revised: As such, the agent can ﬁnish inlier matching timely when the afﬁnity score stops growing, for which otherwise an additional parameter i.e. the number of inliers is needed to avoid matching outliers.

Original: In this paper, we focus on learning the back-end solver under the most general form of GM:  Lawler’s
Revised: In this paper, we focus on learning the back-end solver under the most general form of GM: the Lawler’s

Original: QAP, whose input is the afﬁnity matrix.
Revised: QAP, whose input is the afﬁnity matrix.

Original: Especially, our approach can also boost existing GM methods that use such input.
Revised: Especially, our approach can also boost existing GM methods that use such input.

Original: Experiments on multiple real-world datasets demonstrate its performance regarding both accuracy and robustness.
Revised: Experiments on multiple real-world datasets demonstrate its performance regarding both accuracy and robustness.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph matching (GM) aims to ﬁnd node correspondence between two or multiple graphs.
Revised: Graph matching (GM) aims to ﬁnd node correspondence between two or multiple graphs.

Original: As a standing and fundamental problem, GM spans wide applications in different areas including computer vision and pattern recognition.
Revised: As a standing and fundamental problem, GM spans wide applications in different areas including computer vision and pattern recognition.

Original: With the increasing computing resource, graph matching that involves the second-order edge afﬁnity (in contrast to the linear assignment problem e.g. bipartite matching) becomes a powerful and relatively affordable tool for solving the correspondence problem with moderate size, and there is growing research in this area, especially with the introduction of deep learning in recent years (Zanﬁr et al., 2018;
Revised: With the increasing computing resource, graph matching that involves the second-order edge afﬁnity (in contrast to the linear assignment problem e.g. bipartite matching) becomes a powerful and relatively affordable tool for solving the correspondence problem with moderate size, and there is growing research in this area, especially with the introduction of deep learning in recent years (Zanﬁr et al., 2018;

Original: Wang et al., 2019b).
Revised: Wang et al., 2019b).

Original: In its general form, GM can be formulated as a combinatorial optimization problem namely Lawler’s Quadratic Assignment Problem (Lawler’s QAP)
Revised: GM can be formulated as a combinatorial optimization problem namely Lawler’s Quadratic Assignment Problem (Lawler’s QAP)

Original: (Lawler, 1963), which is known as NP-hard.
Revised: (Lawler, 1963), which is known as NP-hard.

Original: Generally speaking, handling the graph matching problem involves two steps: extracting features from input images to formulate a QAP instance and solving that  instance via constrained optimization, namely front-end feature extractor and back-end solver, respectively.
Revised: Generally speaking, handling the graph matching problem involves two steps: extracting features from input images to formulate a QAP instance and solving that QAP instance via constrained optimization, namely front-end feature extractor and back-end solver, respectively.

Original: Impressive progress has been made for graph matching with the introduction of rich deep learning techniques.
Revised: Impressive progress has been made for graph matching with the introduction of rich deep learning techniques.

Original: However, in existing deep GM works, the deep learning modules are mainly applied on the front-end, especially for visual images using CNN for node feature learning (Zanﬁr et al., 2018) and GNN for structure embedding (Li et al., 2019).
Revised: However, in existing deep GM works, the deep learning modules are mainly applied on the front-end, especially for visual images using CNN for node feature learning (Zanﬁr et al., 2018) and GNN for structure embedding (Li et al., 2019).

Original: Compared withtraditional learning-free methods, learnable features have shown more expressive tailored to the training data.
Revised: Compared with learning-free methods, learnable features have shown more effectiveness.

Original: Anotherimportant advantage of using neural networks is that the graph structure information can be readily embedded into unary node features, as such the classic NP-hard QAP in fact can degenerate into the linear assignment problem, which can be readily solved by existing back-end solvers in polynomial time.
Revised: Another advantage of using neural networks is that the graph structure information can be readily embedded into unary node features, as such the classic NP-hard QAP in fact can degenerate into the linear assignment problem, which can be readily solved by existing back-end solvers in polynomial time.

Original: Perhaps for this reason, existing deep GM methods (Wang et al., 2019b; Fey et al., 2020a;Yu et al., 2020; Jiang et al., 2021a; Zhao et al., 2021; Gao et al., 2021) mostly focus on the front-end learning, basically by supervised learning using manually labeled node correspondence as ground truth.
Revised: Perhaps for this reason, existing deep GM methods (Wang et al., 2019b; Fey et al., 2020a; Zhao et al., 2021; Gao et al., 2021) mostly focus on the front-end learning, basically by supervised learning using manually labeled node correspondence as ground truth.

Original: While the back-end solver is relatively little considered for learning in literature: the authors simply combine their front-end feature extractors with some traditional combinatorial solvers e.g. Sinkhorn (Cuturi, 2013), which means they hardly utilize deep learning to improve the back-end solvers.
Revised: While the back-end solver is relatively little considered for learning in literature: the authors simply combine their front-end feature extractors with some traditional combinatorial solvers e.g. Sinkhorn (Cuturi, 2013), which means they hardly utilize deep learning to improve the back-end solvers.

Original: Despite the above progress made in deep graph matching, little care is taken for dealing with the presence of outliers which is ubiquitous in the real world.

Original: Note that outliers in this paper refer to the common setting in literature (Yang et al., 2015) which are also ubiquitous in real-world matching scenarios: namely the spurious nodes that cannot ﬁnd their correspondence in the opposite graph for matching.
Revised: Note that outliers in this paper refer to the common setting in literature (Yang et al., 2015): namely the spurious nodes that cannot ﬁnd their correspondence in the opposite graph for matching, which are also ubiquitous in real-world matching scenarios.

Original: While inliers are those having correspondences.
Revised: While inliers are those having correspondences.

Original: More speciﬁcally, we assume the most general and challenging case that outliers could exist in both two input graphs, which is in contrast to the majority of works (Zanﬁr et al., 2018; Wang et al., 2021a) that assume there is at most one graph containing outliers.
Revised: More speciﬁcally, we assume the most general and challenging case that outliers could exist in both two input graphs, which is in contrast to the majority of works (Zanﬁr et al., 2018; Wang et al., 2021a) that assume there is at most one graph containing outliers.

Original: Though there is a line of works for image matching effectively dismissing the outliers (Fischler & Bolles, 1981; Ma et al., 2020), these works are basically based on speciﬁc pose and motion models in vision, which may not always be available for general GM tasks.
Revised: Though there is a line of works for image matching effectively dismissing the outliers (Fischler & Bolles, 1981; Ma et al., 2020), these works are basically based on speciﬁc pose and motion models in vision, which may not always be available for general GM tasks.

Original: Moreover, as mentioned above, existing deep GM works are all supervised (or based on supervised learning modules), while in the real world, the labeling is costly and even almost impossible to obtain for large-scale QAP instances in practice.
Revised: Moreover, as mentioned above, existing deep GM works are all supervised (or based on supervised learning modules), while in the real world, the labeling is costly and even almost impossible to obtain for large-scale QAP instances in practice.

Original: Towards practical and robust graph matching learning, in the absence of labels and in the presence of outliers (in both input graphs), we propose a reinforcement learning (RL) method for graph matching namely RGM , especially for its most general QAP formulation.
Revised: Towards practical and robust graph matching learning, in the absence of labels and in the presence of outliers (in both input graphs), we propose a reinforcement learning (RL) method for graph matching namely RGM , especially for its most general QAP formulation.

Original: In particular, RL is conceptually well suited for its label-free nature and ﬂexibility in ﬁnding the node correspondence by sequential decision making, which provides a direct way of avoiding outlier over-matching by an early stopping.
Revised: In particular, RL is conceptually well suited for its label-free nature and ﬂexibility in ﬁnding the node correspondence by sequential decision making, which provides a direct way of avoiding outlier over-matching by an early stopping.

Original: In contrast, in existing deep GM works, matching is performed in one shot which incurs coupling of the inliers and outliers, and it lacks an explicit way to distinguish outliers.
Revised: In contrast, in existing deep GM works, matching is performed in one shot which incurs coupling of the inliers and outliers, and it lacks an explicit way to distinguish outliers.

Original: Therefore, we speciﬁcally devise a so-called revocable deep reinforcement learning framework to allow small mistakes over the matching procedure, and the current action is revocable to research a better node correspondence based on up-to-date environment information.
Revised: Therefore, we speciﬁcally devise a so-called revocable deep reinforcement learning framework to allow small mistakes over the matching procedure, and the current action is revocable to research a better node correspondence based on up-to-date environment information.

Original: Our revocable framework is shown cost-effective and empirically outperforms the existing popular techniques for reﬁning the local decision making e.g. Local Rewrite (Chen & Tian, 2019).
Revised: Our revocable framework is shown cost-effective and empirically outperforms the existing popular techniques for reﬁning the local decision making e.g. Local Rewrite (Chen & Tian, 2019).

Original: Moreover, since the standard GM objective refers to maximizing the afﬁnity score between matched nodes, it causes the over-matching issue i.e. the outliers are also incorporated for matching to increase the overall score.
Revised: Moreover, since the standard GM objective refers to maximizing the afﬁnity score between matched nodes, it causes the over-matching issue i.e. the outliers are also incorporated for matching to increase the overall score.

Original: To address this issue, we propose to regularize the afﬁnity score such that it discourages unwanted matchings by assigning a negative score to those pairs.
Revised: To address this issue, we propose to regularize the afﬁnity score such that it discourages unwanted matchings by assigning a negative score to those pairs.

Original: Intuitively, the RL agent will naturally stop matching spurious outliers as the objective score will otherwise decrease.
Revised: Intuitively, the RL agent will naturally stop matching spurious outliers as the objective score will otherwise decrease.

Original: With the help of the revocable framework and afﬁnity regularization, our RGM shows promising performances in various experiments.
Revised: With the help of the revocable framework and afﬁnity regularization, our RGM shows promising performances in various experiments.

Original: For more clearance, we compare our RGM with most of the existing GM methods in Table 5.
Revised: For more clearance, we compare our RGM with most of the existing GM methods in Table 5.

Original: Due to the space limit, we place it in the appendix (A.1), where we add a more detailed discussion of the comparison of RGM and existing works.
Revised: Due to the space limit, we place it in the appendix (A.1), where we add a more detailed discussion of the comparison of RGM and existing works.

Original: To sum up, the highlights and contributions of our work are:
Revised: To sum up, the highlights and contributions of our work are:

Original: 1) We propose RGM that sequentially selects the node correspondences from two graphs, in contrast to the majority of existing works that obtain the whole matching in one shot.
Revised: 1) We propose RGM that sequentially selects the node correspondences from two graphs, in contrast to the majority of existing works that obtain the whole matching in one shot.

Original: Accordingly, our approach can naturally handle the case for partial matching (due to outliers) by early stopping.
Revised: Accordingly, our approach can naturally handle the case for partial matching (due to outliers) by early stopping.

Original: 2) Speciﬁcally, we ﬁrst devise a revocable approach to select the possible node correspondence, whose mechanism is adapted to the unlabeled graph data with the afﬁnity score as the reward.
Revised: 2) Speciﬁcally, we ﬁrst devise a revocable approach to select the possible node correspondence, whose mechanism is adapted to the unlabeled graph data with the afﬁnity score as the reward.

Original: To the best of our knowledge, this is the ﬁrst attempt to successfully adapt RL to graph matching.
Revised: To the best of our knowledge, this is the ﬁrst attempt to successfully adapt RL to graph matching.

Original: For avoiding matching the outliers, we develop a regularization to the afﬁnity score, making the solver no longer pursue to match as many nodes as possible.
Revised: For avoiding matching the outliers, we develop a regularization to the afﬁnity score, making the solver no longer pursue to match as many nodes as possible.

Original: To our best knowledge, this is also the ﬁrst work for regularizing the afﬁnity score to avoid over-matching among outliers.
Revised: To our best knowledge, this is also the ﬁrst work for regularizing the afﬁnity score to avoid over-matching among outliers.

Original:  On synthetic datasets, Willow Object dataset, Pascal VOC dataset, and QAPLIB, RGM shows competitive performance compared with both learning-free and learning-based baselines.
Revised: 4) On synthetic datasets, Willow Object dataset, Pascal VOC dataset, and QAPLIB datasets, RGM shows competitive performance compared with both learning-free and learning-based baselines.

Original: Note that RGM focuses on learning the back- end solver and hence it is orthogonal to many existing frontend feature learning based GM methods, which can further boost the front-end learning solvers’ performance as shown in our experiments.
Revised: Note that RGM focuses on learning the back-end solver and hence it is orthogonal to many existing front-end feature learning based GM methods, which can further boost the front-end learning solvers’ performance as shown in our experiments.

Original: The source code will be made public after the publication.

Original:  R ELATED W ORK
Revised:  R ELATED W ORKS

Original: Graph matching.
Revised: Graph matching.

Original: In this paper, we consider graph matching, that aims to ﬁnd the node correspondence by considering both the node features as well as the edge attributes, which is known as NP-hard in its general form (Yan et al., 2020; Loiola et al., 2007).
Revised: It aims to ﬁnd the node correspondence by considering both the node features as well as the edge attributes, which is known as NP-hard in its general form (Loiola et al., 2007).

Original: Classic methods mainly resort to different optimization heuristics ranging from random walk (Cho et al., 2010), spectral match- name
Revised: Classic methods mainly resort to different optimization heuristics e.g. random walk (Cho et al., 2010), spectral matching (Leordeanu & Hebert, 2005), path-following (Zhou & Torre, 2016), graduated assignment (Gold et al., 1996), to SDP relaxation (Schellewald & Schn¨orr, 2005).

Original: Deep learning of GM.
Revised: Deep learning of GM.

Original: Since the seminal work (Zanﬁr et al., 2018), deep neural networks have been an promising paradigm forsolving combinatorial problems (Bengio et al., 2021) including graph matching (Yan et al., 2020).
Revised: Since the seminal work (Zanﬁr et al., 2018), deep neural networks have been devised for graph matching (Yan et al., 2020).

Original: Among the deep GM methods, we generally discuss two representative lines of research as follow.
Revised: Among the deep GM methods, we generally discuss two representative lines of research as follow.

Original: The ﬁrst line of works (Wang et al., 2019b; Yu et al., 2020) apply CNNs or/and GNNs for learning the node and structure features.
Revised: The ﬁrst line of works (Wang et al., 2019b; Yu et al., 2020) apply CNNs or/and GNNs for learning the node and structure features.

Original: By using node embedding, the problem degrades into linear assignment that can be optimally solved by the Sinkhorn network (Cuturi, 2013) to fulﬁll double-stochasticity which is non-learnable.
Revised: By using node embedding, the problem degrades into linear assignment that can be optimally solved by the Sinkhorn network (Cuturi, 2013) to fulﬁll double-stochasticity which is non-learnable.

Original: Instead, another line of studies (Wang et al., 2020c) follow the general Lawler’s QAP form exactly, and the problem becomes combinatorial selecting of nodes on the association graphs.
Revised: Instead, another line of studies (Wang et al., 2020c) follow the general Lawler’s QAP form exactly, and the problem becomes combinatorial selecting of nodes on the association graphs.

Original: Yet all the above models adopt supervised learning and there is little reported success using RL to solve GM despite their label-free advantage and popularity in solving other combinatorial problems.
Revised: Yet all the above models adopt supervised learning and there is little reported success using RL to solve GM despite their label-free advantage and popularity in solving other combinatorial problems.

Original: Matching against outliers.
Revised: Matching against outliers.

Original: Matching against outliers has been a long standing task especially for vision, and seminal works date back to RANSAC (Fischler & Bolles, 1981).
Revised: Matching against outliers has been a long standing task especially for vision, and seminal works date back to RANSAC (Fischler & Bolles, 1981).

Original: There are efforts (Torresani et al., 2012; Yang et al., 2015;
Revised: There are efforts (Torresani et al., 2012; Yang et al., 2015;

Original: Yi et al., 2018; Zhang et al., 2019; Liu et al., 2020) in exploring the speciﬁc problem structure and clues in terms of spatial and geometry coherence to achieve robust matching.
Revised: Yi et al., 2018; Zhang et al., 2019; Liu et al., 2020) in exploring the speciﬁc problem structure and clues in terms of spatial and geometry coherence to achieve robust matching.

Original: While we focus on the general setting of GM without using additional assumptions or parametric transform models, the relevant pair-wise graph matching works are relatively less crowded.
Revised: While we focus on the general setting of GM without using additional assumptions or parametric transform models, the relevant pair-wise graph matching works are relatively less crowded.

Original: Wang et al.
Revised: Wang et al.

Original: (2019a) utilizes the strategy based on domain adaptation, which removes the outliers by data normalization module.
Revised: (2019a) utilizes the strategy based on domain adaptation, which removes the outliers by data normalization module.

Original: A heuristic max-pooling strategy is developed in Cho et al.
Revised: A heuristic max-pooling strategy is developed in Cho et al.

Original: (2014) for dismissing excessive outliers.
Revised: (2014) for dismissing excessive outliers.

Original: Wang et al.
Revised: Wang et al.

Original: (2020a) proposes to suppress the matching of outliers by assigning zero-valued vectors to the potential outliers.
Revised: (2020a) proposes to suppress the matching of outliers by assigning zero-valued vectors to the potential outliers.

Original: However, all the above methods are learning-free and the trending learning-based solvers to our best knowledge, have not addressed the outlier problems explicitly except for adding dummy nodes (Wang et al., 2021a).
Revised: However, all the above methods are learning-free and the trending learning-based solvers to our best knowledge, have not addressed the outlier problems explicitly except for adding dummy nodes (Wang et al., 2021a).

Original: Reinforcement learning for combinatorial optimization.
Revised: Reinforcement learning for combinatorial optimization.

Original: There is growing interest in using RL in solving combinatorial optimization problems (Bengio et al., 2021), such as traveling salesman problem (Khalil et al., 2017), vehicle routing problem (Nazari et al., 2018), job scheduling problem (Chen & Tian, 2019), maximal common subgraph (Bai et al., 2021), The main challenges of these approaches are designing suitable problem representation and tuning reinforcement learning algorithms.
Revised: There is growing interest in using RL in solving combinatorial optimization problems (Bengio et al., 2021), such as traveling salesman problem (Khalil et al., 2017), vehicle routing problem (Nazari et al., 2018), job scheduling problem (Chen & Tian, 2019), maximal common subgraph (Bai et al., 2021), The main challenges of these approaches are designing suitable problem representation and tuning reinforcement learning algorithms.

Original: For combinatorial optimization problems on single graph, pointer networks (Vinyals et al., 2015) and GNN (Kipf & Welling, 2017) are the most widely used representations.
Revised: For combinatorial optimization problems on single graph, pointer networks (Vinyals et al., 2015) and GNN (Kipf & Welling, 2017) are the most widely used representations.

Original: However, for graph matching, there are two graphs for input and the agent needs to pick a node from each of the two graphs every step.
Revised: However, for graph matching, there are two graphs for input and the agent needs to pick a node from each of the two graphs every step.

Original: To our best knowledge, GM has not been (successfully) addressed by RL.
Revised: To our best knowledge, GM has not been (successfully) addressed by RL.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: In this paper, we mainly focus on two graph matching i.e. pairwise graph matching, which is also known as pairwise graph matching.
Revised: In this paper, we mainly focus on two graph matching i.e. pairwise graph matching, which is also known as pairwise graph matching.

Original: Speciﬁcally, we consider a more difﬁcult situation, where there are some outliers in both two graphs, and we want to match the similar inliers while ignoring outliers.
Revised: Speciﬁcally, we consider a more difﬁcult situation, where there are some outliers in both two graphs, and we want to match the similar inliers while ignoring outliers.

Original: Given two weighted graphs G 1 and G 2 , we aim to ﬁnd the matching between their nodes such that the afﬁnity score is maximized.
Revised: Given two weighted graphs G 1 and G 2 , we aim to ﬁnd the matching between their nodes such that the afﬁnity score is maximized.

Original: We use V 1 and V 2 to represent the nodes of graph G 1 and G 2 .
Revised: We use V 1 and V 2 to represent the nodes of graph G 1 and G 2 .

Original: We suppose that | V 1 | = n 1 , | V 2 | = n 2 , and there can be outliers in G 1 , G 2 , or both .
Revised: We suppose that | V 1 | = n 1 , | V 2 | = n 2 , and there can be outliers in G 1 , G 2 , or both .

Original: E 1 and Edenote the edge attributes of graph G 1 and G 2 .
Revised: E 1 and Edenote the edge attributes of graph G 1 and G 2 .

Original: The afﬁnities of pairwise graph matching include the ﬁrst order (node) afﬁnities and the second order (edge) afﬁnities.
Revised: The afﬁnities of pairwise graph matching include the ﬁrst order (node) afﬁnities and the second order (edge) afﬁnities.

Original: Generally, the graph matching problem can be regarded as Lawler’s Quadratic Assignment Problem (Lawler, 1963):
Revised: Generally, the graph matching problem can be regarded as Lawler’s Quadratic Assignment Problem (Lawler, 1963):

Original: [Equation]
Revised: [Equation]

Original: where X is the permutation matrix of which the element is 0 or 1 , X
Revised: where X is the permutation matrix of which the element is 0 or 1 , X

Original: ( i, a ) = 1 denotes node i in graph G 1 is matched with node a in graph G 2 .
Revised: ( i, a ) = 1 denotes node i in graph G 1 is matched with node a in graph G 2 .

Original: The operator vec ( · ) means column-vectorization.
Revised: The operator vec ( · ) means column-vectorization.

Original: K ∈ R n 1 n 2 × n 1 n 2 is the afﬁnity matrix.
Revised: K ∈ R n 1 n 2 × n 1 n 2 is the afﬁnity matrix.

Original: For node i in G 1 and node a in G 2 the node-to-node afﬁnity is encoded by the diagonal element K ( ia, ia ) , while for edge ij in G 1 and edge ab in G 2 the edgeto-edge afﬁnity is encoded by the off-diagonal element K ( ia, jb ) .
Revised: For node i in G 1 and node a in G 2 the node-to-node afﬁnity is encoded by the diagonal element K ( ia, ia ) , while for edge ij in G 1 and edge ab in G 2 the edgeto-edge afﬁnity is encoded by the off-diagonal element K ( ia, jb ) .

Original: Assuming i, a both start from 0,[-the index ia means i × n 
 + a .-]
Revised: Assuming i, a both start from 0,[-the index ia means i × n 
 + a .-]

Original: The objective is to maximize the sum of both ﬁrst order and second order afﬁnity score J ( X ) given the afﬁnity matrix K by ﬁnding an optimal permutation X .

Original: a b c 1c 2b 1a 2c
Revised: a b c 1c 2b 1a 2c

Original: K 1a,2b K 1a,1a
Revised: K 1a,2b K 1a,1a

Original: K 2b,2b 1b 2a 3b 3a 3c
Revised: K 2b,2b 1b 2a 3b 3a 3c

Original: G
Revised: G

Original: G
Revised: G

Original: G a  a b c 3 G
Revised: G a  a b c 3 G

Original: G  a b c  a b c  a b c 3 G
Revised: G  a b c  a b c  a b c 3 G

Original: G
Revised: G

Original: G G
Revised: G G

Original: G
Revised: G

Original: G1c 2b 1a 2c 1b 2a 3b 3a 3c
Revised: G1c 2b 1a 2c 1b 2a 3b 3a 3c

Original: G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c
Revised: G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c

Original: G a 1c 2b 1a 2c 1b 2a 3b 3a 3c
Revised: G a 1c 2b 1a 2c 1b 2a 3b 3a 3c

Original: G a 
Revised: G a the index ia means i × n 2 + a . The objective is to maximize the sum of both ﬁrst order and second order afﬁnity score J ( X ) given the afﬁnity matrix K by ﬁnding an optimal permutation X .

Original: Graph matching involves (at least) two input graphs.
Revised: Graph matching involves (at least) two input graphs.

Original: Instead of directly working on two individual graphs which can disclose raw data information and might be sensitive, in this paper, we ﬁrst construct the association graph of the pairwise graphs as the input representation (Leordeanu & Hebert, 2005; Wang et al., 2021a).
Revised: Instead of directly working on two individual graphs which can disclose raw data information and might be sensitive, in this paper, we ﬁrst construct the association graph of the pairwise graphs as the input representation (Leordeanu & Hebert, 2005; Wang et al., 2021a).

Original: Speciﬁcally, we construct an association graph G a =
Revised: Speciﬁcally, we construct an association graph G a =

Original: ( V a , E a ) from the original pairwise graph G 1 and G 2 , with the help of the afﬁnity matrix K .
Revised: ( V a , E a ) from the original pairwise graph G 1 and G 2 , with the help of the afﬁnity matrix K .

Original: We merge each two nodes ( v i , v a ) ∈ V 1 × V 2 as a vertex v p ∈ V a .
Revised: We merge each two nodes ( v i , v a ) ∈ V 1 × V 2 as a vertex v p ∈ V a .

Original: We can see that the association graph contains | V a | = n 1 × n 2 vertices 1 . There exists an edge for every two vertices as long as they do not contain the same node from the original pairwise graphs, so every vertex is connected to ( n 1 − 1) × ( n 2 − 1) edges.
Revised: We can see that the association graph contains | V a | = n 1 × n 2 vertices 1 .There exists an edge for every two vertices as long as they do not contain the same node from the original pairwise graphs, so every vertex is connected to ( n 1 − 1) × ( n 2 − 1) edges.

Original: There exist both vertex weights w ( v p ) and edge weights w ( v p , v q ) in the association graph, where the vertex and edge weights denote the ﬁrst and second order afﬁnities:
Revised: There exist both vertex weights w ( v p ) and edge weights w ( v p , v q ) in the association graph, where the vertex and edge weights denote the ﬁrst and second order afﬁnities:

Original: [Equation]
Revised: [Equation]

Original: wherep = ia [Equation] where p = ia, q = jb (2) where the vertex index p in the association graph G a means a combination of the indices i and a in the original pairwise graphs G 1 and G 2 .
Revised: where the vertex index p in the association graph G a means a combination of the indices i and a in the original pairwise graphs G 1 and G 2 .

Original: F , W ∈ R n 1 n 2 × n 1 n 2 are the weight matrices that contain the vertex weights and edge weights in the association graph, respectively.
Revised: F , W ∈ R n 1 n 2 × n 1 n 2 are the weight matrices that contain the vertex weights and edge weights in the association graph, respectively.

Original: Fig.
Revised: Fig.

Original: 1 shows an example to construct the association graph from the input graphs.
Revised: 1 shows an example to construct the association graph from the input graphs.

Original: In the association graph G a , selecting a vertex p in the association graph equals to matching nodes i and a in the input graphs.
Revised: In the association graph G a , selecting a vertex p in the association graph equals to matching nodes i and a in the input graphs.

Original: Therefore, we can select a set of vertices U as we choose to match these nodes as the solution.
Revised: Therefore, we can select a set of vertices U as we choose to match these nodes as the solution.

Original: Note that the set of vertices U in the association graph is equivalent to the permutation matrix X
Revised: Note that the set of vertices U in the association graph is equivalent to the permutation matrix X

Original: and can be easily converted, as long as the set U does not violate the constraint in Eq.
Revised: and can be easily converted, as long as the set U does not violate the constraint in Eq.

Original:  A PPROACH 4. R EINFORCEMENT L EARNING FOR G RAPH M ATCHING
Revised:  A PPROACH 4. R EINFORCEMENT L EARNING FOR G RAPH M ATCHING

Original: In this paper, we design our RGM based on Double Dueling DQN (D3QN)
Revised: We design our RGM based on Double Dueling DQN (D3QN)

Original: (Hasselt et al., 2016), as it is widely accepted (Sutton & Barto, 2018)
Revised: (Hasselt et al., 2016), as it is widely accepted (Sutton & Barto, 2018)

Original: that value based RL algorithms like D3QN can better handle the discrete case, which will be also veriﬁed in our ablation study.
Revised: that value based RL algorithms like D3QN can better handle the discrete case, which will be also veriﬁed in our ablation study.

Original: The pipeline of the training process of RGM is described in Alg. 1.
Revised: The pipeline of the training process of RGM is described in Alg. 1.

Original: Due to the page limit, several details of the proposed method are shown in the appendix(A.2), includingthe network structure (A.2.1),the prioritized experience replay memory (A.2.2) andthe model updating algorithm (A.2.3).
Revised: Details of the proposed method are given in Appendix (A.2), including network structure (A.2.1), prioritized experience replay memory (A.2.2) and model updating algorithm (A.2.3).

Original: Here, we show the design of state, action, and reward in our RGM: 1) State.
Revised: We show the design of state, action, and reward in our RGM: 1) State.

Original: The state s is the current (partial) solution U (cid:48) , where U (cid:48) is also a set of vertices in the association graph, with | U (cid:48) | ≤ min ( n 1 , n 2 ) .
Revised: The state s is the current (partial) solution U (cid:48) , where U (cid:48) is also a set of vertices in the association graph, with | U (cid:48) | ≤ min ( n 1 , n 2 ) .

Original: The size of U (cid:48) increases from 0 at the beginning, and ﬁnally partial solution U (cid:48) becomes a complete solution U when the agent decides to stop the episode.
Revised: The size of U (cid:48) increases from 0 at the beginning, and ﬁnally partial solution U (cid:48) becomes a complete solution U when the agent decides to stop the episode.

Original: 2) Action.
Revised: 2) Action.

Original: The action a of our reinforcement learning agent is to select a vertex in the association graph and add it to the current solution U (cid:48) .
Revised: The action a of our reinforcement learning agent is to select a vertex in the association graph and add it to the current solution U (cid:48) .

Original: By the deﬁnition of graph matching, we can not match two nodes in G 1 to one same node in G 2 and vice versa.
Revised: By the deﬁnition of graph matching, we can not match two nodes in G 1 to one same node in G 2 and vice versa.

Original: Therefore, in our basic RL framework, we can only select the vertices in the available vertex set.
Revised: Therefore, in our basic RL framework, we can only select the vertices in the available vertex set.

Original: Take Fig.
Revised: Take Fig.

Original: 1 for an example, once we select 
Revised: 1 for an example, once we select the vertex “1a”, it means we have matched node “1” in G 1 and node “a” in G 2 . Then, we can not

Original: 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a
Revised: 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: select undo 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a Action
Revised: select undo 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a Action

Original: Action
Revised: Action

Original: Action1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a Actionselect undo undo 1c 2b 1a 2c 1b 2a 3b 3a 3c G a Action
Revised: Action1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a Actionselect undo undo 1c 2b 1a 2c 1b 2a 3b 3a 3c G a Action

Original: A
Revised: A

Original: B
Revised: B

Original: C 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G athe vertex “1a”, it means we have matched node “1” in G 1 and node “a” in G 2 . Then, we can not match node “1” to node “b” or “c” later, which means we can not select vertices “1b” or “1c”.
Revised: C 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a 1c 2b 1a 2c 1b 2a 3b 3a 3c G a match node “1” to node “b” or “c” later, which means we can not select vertices “1b” or “1c”.

Original: Given partial solution U (cid:48) , the available vertices set V is written as:
Revised: Given partial solution U (cid:48) , the available vertices set V is written as:

Original: [Equation]
Revised: [Equation]

Original: where V a is the vertices in the association graph, whose adjacency matrix (0 for no edge connected and 1 for existing edge connected) is A .
Revised: where V a is the vertices in the association graph, whose adjacency matrix (0 for no edge connected and 1 for existing edge connected) is A .

Original: Eq.
Revised: Eq.

Original: 3 holds since two vertices are connected if they do not contain the same node from the input graphs.
Revised: 3 holds since two vertices are connected if they do not contain the same node from the input graphs.

Original: If a vertex is connected to all vertices in U (cid:48) , then it has no conﬂict.
Revised: If a vertex is connected to all vertices in U (cid:48) , then it has no conﬂict.

Original: Given the available set, we mask all unavailable vertices in the association graph to make sure that the agent can not select them, as illustrated by the blurred vertices in Fig.
Revised: Given the available set, we mask all unavailable vertices in the association graph to make sure that the agent can not select them, as illustrated by the blurred vertices in Fig.

Original: Then, the action is to pick a node v from the available vertices set V : U old v ∈ V −−→ U new , where U old is the old partial solution, and U new is the new partial solution after an action.
Revised: Then, the action is to pick a node v from the available vertices set V : U old v ∈ V −−→ U new , where U old is the old partial solution, and U new is the new partial solution after an action.

Original: It is worth noting that, the requirement that the agent needs to select the vertex from the available set only exists in our basic RL framework.
Revised: It is worth noting that, the requirement that the agent needs to select the vertex from the available set only exists in our basic RL framework.

Original: In our later proposed revocable RL framework, the agent can select any vertex without constraint.
Revised: In our later proposed revocable RL framework, the agent can select any vertex without constraint.

Original: 3) Reward.
Revised: 3) Reward.

Original: We deﬁne the reward r as the improvement of the objective score between the old partial solution and the new partial solution after executing an action.
Revised: We deﬁne the reward r as the improvement of the objective score between the old partial solution and the new partial solution after executing an action.

Original: 4. T HE R EVOCABLE A CTION M ECHANISM
Revised: 4. T HE R EVOCABLE A CTION M ECHANISM

Original: So far we have presented a basic RL framework.
Revised: So far we have presented a basic RL framework.

Original: In such a vanilla form, it can not undo the actions that have been executed.
Revised: In such a vanilla form, it can not undo the actions that have been executed.

Original: In other words, the agent can not “regret”, which means the agent has no chance to adjust if the agent makes a wrong decision and the error may accumulate until obtaining a disastrous solution.
Revised: In other words, the agent can not “regret”, which means the agent has no chance to adjust if the agent makes a wrong decision and the error may accumulate until obtaining a disastrous solution.

Original: To strike a reasonable trade-off between efﬁciency and efﬁcacy, we develop a mechanism to allow the agent to re-select the vertex on the association in one revocable step.
Revised: To strike a reasonable trade-off between efﬁciency and efﬁcacy, we develop a mechanism to allow the agent to re-select the vertex on the association in one revocable step.

Original: To allow the agent to modify the decisions made before, we design a new revocable RL framework.
Revised: To allow the agent to modify the decisions made before, we design a new revocable RL framework.

Original: We remove the available set used before, and the agent is free to choose any vertex, even if it is in conﬂict.
Revised: We remove the available set used before, and the agent is free to choose any vertex, even if it is in conﬂict.

Original: Then, we modify the strategy in our RL environment.
Revised: Then, we modify the strategy in our RL environment.

Original: If the environment receives a new vertex from the agent’s action that is in conﬂict with currently selected vertices, the environment will remove one or two vertices that are in conﬂict with the new coming vertex, and then add the new vertex to the current solution.
Revised: If the environment receives a new vertex from the agent’s action that is in conﬂict with currently selected vertices, the environment will remove one or two vertices that are in conﬂict with the new coming vertex, and then add the new vertex to the current solution.

Original: Our proposed revocable RL framework is illustrated in Fig.
Revised: Our proposed revocable RL framework is illustrated in Fig.

Original: As Fig.
Revised: As Fig.

Original: 2 shows, the pipelines of our revocable RGM are a little different from our basic framework.
Revised: 2 shows, the pipelines of our revocable RGM are a little different from our basic framework.

Original: The available set in the basic RGM does not exist if the revocable mode is on.
Revised: The available set in the basic RGM does not exist if the revocable mode is on.

Original: Pipeline (A) shows a simple situation that the agent matched the vertices directly without any reverse operation.
Revised: Pipeline (A) shows a simple situation that the agent matched the vertices directly without any reverse operation.

Original: In pipeline (B), we suppose the agent chooses the vertex “2c” for the second action, which is not a good choice.
Revised: In pipeline (B), we suppose the agent chooses the vertex “2c” for the second action, which is not a good choice.

Original: When choosing the third action, the agent realizes that it made a mistake by selecting “2c”, therefore, the agent chooses “2b” to ﬁx this mistake.
Revised: When choosing the third action, the agent realizes that it made a mistake by selecting “2c”, therefore, the agent chooses “2b” to ﬁx this mistake.

Original: Then, this action is passed to the environment, and the environment reverts “2c” and selects “2b” instead.
Revised: Then, this action is passed to the environment, and the environment reverts “2c” and selects “2b” instead.

Original: In other words, the agent can reverse “2c” to “2b”.
Revised: In other words, the agent can reverse “2c” to “2b”.

Original: In pipeline (C), we show another revocable situation.
Revised: In pipeline (C), we show another revocable situation.

Original: Suppose that the agent selects 
Revised: Suppose that the agent selects “2c” and “3b” as its second and third actions, and acquires a complete matching solution. However, it turns out the matching solution (“1a”, “2c”, and “3b”) is not as good as expected. To roll this

Original: [Table]
Revised: [Table]

Original: am.xlsx -0.3 0.2 0.1 0.5 0.3 0.1 0.1 0.1 0.4 0.2 -0.4 0.3 0.4 0.

Revised: am.xlsx -0.3 0.2 0.1 0.5 0.3 0.1 0.1 0.1 0.4 0.2 -0.4 0.3 0.4 0.

Original: [Figure]
Revised: [Figure]

Original: “2c” and “3b” as its second and third actions, and acquires a complete matching solution.

Original: However, it turns out the matching solution (“1a”, “2c”, and “3b”) is not as good as expected.

Original: To roll this situation back, the agent can select “2b” as the next action.
Revised: situation back, the agent can select “2b” as the next action.

Original: By selecting “2b”, the environment will release the vertices “2c” and “3b”, and then select “2b”.
Revised: By selecting “2b”, the environment will release the vertices “2c” and “3b”, and then select “2b”.

Original: Finally, the agent chooses “3c” as the last action, and decides to end the episode with the matching solution (“1a”, “2b”, and “3c”).
Revised: Finally, the agent chooses “3c” as the last action, and decides to end the episode with the matching solution (“1a”, “2b”, and “3c”).

Original: Above all, our proposed revocable reinforcement learning framework allows the agent to make better decisions by giving the opportunity to turn the way back.
Revised: Above all, our proposed revocable reinforcement learning framework allows the agent to make better decisions by giving the opportunity to turn the way back.

Original: From now on in the paper, the default setting of RGM contains the revocable action framework.
Revised: From now on in the paper, the default setting of RGM contains the revocable action framework.

Original: For further discussion, especially the difference from other similar frameworks (especially the popular Local Rewrite (Chen & Tian, 2019) and ECO-DQN (Barrett et al., 2020)), please refer to the appendix(A.2.4).
Revised: For further discussion, especially the difference from other similar frameworks (especially the popular Local Rewrite (Chen & Tian, 2019) and ECO-DQN (Barrett et al., 2020)), please refer to the appendix(A.2.4).

Original: 4. O UTLIER -R OBUST G RAPH M ATCHING
Revised: 4. O UTLIER -R OBUST G RAPH M ATCHING

Original: For practical GM, outliers are common in both  input pairwise graphs.
Revised: For practical GM, outliers are common in both the input pairwise graphs.

Original: In general, the solution is supposed to contain only the inlier correspondences without outliers.
Revised: In general, the solution is supposed to contain only the inlier correspondences without outliers.

Original: However, most existing GM methods are designed to match all keypoints whether they are inliers or outliers.
Revised: However, most existing GM methods are designed to match all keypoints whether they are inliers or outliers.

Original: This design is based on pursuing the highest objective score, and matching outliers also can increase the objective score to some extent.
Revised: This design is based on pursuing the highest objective score, and matching outliers also can increase the objective score to some extent.

Original: We believe that the outliers should not be matched in any sense.
Revised: We believe that the outliers should not be matched in any sense.

Original: Therefore, we propose two strategies to guide the agent to match the inliers only and ignore the outliers.
Revised: Therefore, we propose two strategies to guide the agent to match the inliers only and ignore the outliers.

Original: 4.3. I NLIER C OUNT I NFORMATION E XPLORATION
Revised: 4.3. I NLIER C OUNT I NFORMATION E XPLORATION

Original: Our ﬁrst strategy is to inform the agent of the amount of common inliers.
Revised: Our ﬁrst strategy is to inform the agent of the amount of common inliers.

Original: Similar input settings can also be found in learning-free outlier-robust GM methods (Yang et al., 2017; Wang et al., 2020a).
Revised: Similar input settings can also be found in learning-free outlier-robust GM methods (Yang et al., 2017; Wang et al., 2020a).

Original: Given the exact number of inliers n i , the sequential matching of RGM can be readily stopped when the size of the current solution | U (cid:48) | = n i , leaving the remaining n 1 − n i and n 2 − n i nodes as outliers.
Revised: Given the exact number of inliers n i , the sequential matching of RGM can be readily stopped when the size of the current solution | U (cid:48) | = n i , leaving the remaining n 1 − n i and n 2 − n i nodes as outliers.

Original: 4.3. A FFINITY R EGULARIZATION VIA Q UADRATIC A PPROXIMATION
Revised: 4.3. A FFINITY R EGULARIZATION VIA Q UADRATIC A PPROXIMATION

Original: Our second strategy is to regularize the afﬁnity score (i.e. objective score).
Revised: Our second strategy is to regularize the afﬁnity score (i.e. objective score).

Original: As mentioned before, existing GM methods tend to match all keypoints including outliers to make the afﬁnity score as high as possible.
Revised: As mentioned before, existing GM methods tend to match all keypoints including outliers to make the afﬁnity score as high as possible.

Original: Therefore, one straightforward idea is to regularize the afﬁnity score which exerts penalization on over-matching terms to balance the effect of the outliers.
Revised: Therefore, one straightforward idea is to regularize the afﬁnity score which exerts penalization on over-matching terms to balance the effect of the outliers.

Original: In this paper, we propose to regularize the original afﬁnity score that blindly adds up all the node/edge correspondence afﬁnity values including the outliers.
Revised: In this paper, we propose to regularize the original afﬁnity score that blindly adds up all the node/edge correspondence afﬁnity values including the outliers.

Original: In other words, we aim to design an inlier-aware regularized afﬁnity score denoted by J reg ( X ) to dismiss the effect of outliers in afﬁnity score computing, as the afﬁnity score from the inlier matching is more meaningful.
Revised: In other words, we aim to design an inlier-aware regularized afﬁnity score denoted by J reg ( X ) to dismiss the effect of outliers in afﬁnity score computing, as the afﬁnity score from the inlier matching is more meaningful.

Original: Speciﬁcally, the regularization is devised as a function w.r.t.
Revised: Speciﬁcally, the regularization is devised as a function w.r.t.

Original: the number of matched keypoints in X , as denoted by f ( (cid:107) vec ( X ) (cid:107) 1 ) , which is multiplied on the original afﬁnity score as follows:[Equation]
Revised: the number of matched keypoints in X , as denoted by f ( (cid:107) vec ( X ) (cid:107) 1 ) , which is multiplied on the original afﬁnity score as follows:

Original: [Equation]
Revised: [Equation]

Original: In general, f ( (cid:107) vec ( X ) (cid:107) 1 ) shall become smaller when there are more matched keypoints to suppress spurious outliers.
Revised: In general, f ( (cid:107) vec ( X ) (cid:107) 1 ) shall become smaller when there are more matched keypoints to suppress spurious outliers.

Original: For the effectiveness, in this paper we choose the following three functions without loss of generality: f 1 ( n ) =
Revised: For the effectiveness, in this paper we choose the following three functions without loss of generality: f 1 ( n ) =

Original: 3max( n 1 ,n 2 ) − n 3max( n 1 ,n 2 ) ,f [Equation] n [Equation]
Revised: 3max( n 1 ,n 2 ) − n 3max( n 1 ,n 2 ) ,

Original: =

Original: [Equation]

Original: +

Original: [Equation]

Original: +

Original: [Equation]
Revised: [Equation]

Original: , f

Original: [Equation] = [Equation] However, the above formula is unfriendly for learning in RGM as our core network GNN can only accept the form of the afﬁnity matrix K namely the association graph as input, while the impact of f ( (cid:107) vec ( X ) (cid:107) 1 ) can not be delivered to the GNN.
Revised: However, the above formula is unfriendly for learning in RGM as our core network GNN can only accept the form of the afﬁnity matrix K namely the association graph as input, while the impact of f ( (cid:107) vec ( X ) (cid:107) 1 ) can not be delivered to the GNN.

Original: To ﬁll this gap, in the following, we further propose a technique to transform Eq. 4 into a standard QAP formulation, and construct the regularized afﬁnity matrix K reg as the input for the GNN, instead of the original afﬁnity matrix K .[Figure] n
Revised: To ﬁll this gap, in the following, we further propose a technique to transform Eq. 4 into a standard QAP formulation, and construct the regularized afﬁnity matrix K reg as the input for the GNN, instead of the original afﬁnity matrix K .

Original: [Figure] [Figure] [Figure] [Figure]
Revised: Recall the original score in Eq.

Original: [Figure] [Figure] [Figure] Recall the original score J = vec ( X ) (cid:62) K vec ( X ) andhere we further denote n x = (cid:107) vec ( X ) (cid:107) 1 .
Revised: and we further denote n x = (cid:107) vec ( X ) (cid:107) 1 .

Original: Eq.
Revised: Eq.

Original: 4 can be rewritten as:
Revised: 4 can be rewritten as:

Original: [Equation]
Revised: [Equation]

Revised: [Figure]

Revised: [Figure]

Original:  In the above formula, it is worth noting that to make the above formula a quadratic one, we introduce a quadratic function: g ( n ) = an 2 + bn + c , which is used to approximate the term g ( n ) ≈ 1 − f ( n ) , in a way of least square ﬁtting to determine the unknown coefﬁcients a, b, c .
Revised: [Figure] [Figure] [Figure] In the above formula, it is worth noting that to make the above formula a quadratic one, we introduce a quadratic function: g ( n ) = an 2 + bn + c , which is used to approximate the term g ( n ) ≈ 1 − f ( n ) , in a way of least square ﬁtting to determine the unknown coefﬁcients a, b, c .

Original: Technically the ﬁtting involves sampling n in a certain range according to the prior of matching problem at hand, e.g. n ∈ { 7 , 8 , 9 , 10 , 11 , 12 } .
Revised: Technically the ﬁtting involves sampling n in a certain range according to the prior of matching problem at hand, e.g. n ∈ { 7 , 8 , 9 , 10 , 11 , 12 } .

Original: Then, g ( n ) can be expanded by: ( K A = 1 (all-one matrix) and K B = I )
Revised: Then, g ( n ) can be expanded by: ( K A = 1 (all-one matrix) and K B = I )

Original: [Equation]
Revised: [Equation]

Original: During the iteratively problem solving in RGM, we note that J changes relatively slowly between two consecutive iterations, and thus we treat it as a constant during iteration.
Revised: During the iteratively problem solving in RGM, we note that J changes relatively slowly between two consecutive iterations, and thus we treat it as a constant during iteration.

Original: Based on this observation, we then try to approximately convert J reg into a QAP formulation as follows:
Revised: Based on this observation, we then try to approximately convert J reg into a QAP formulation as follows:

Original: [Equation]
Revised: [Equation]

Original: which is friendly to GNN learning.
Revised: which is friendly to GNN learning.

Original: Let K reg =
Revised: Let K reg =

Original: K − aJ · K A − bJ · K B and Eq.
Revised: K − aJ · K A − bJ · K B and Eq.

Original: 5 becomes:
Revised: 5 becomes:

Original: [Equation]
Revised: [Equation]

Original: In this way, we now can input the regularized afﬁnity matrix K reg to the GNN, to better learn the impact of the regularization term f ( (cid:107) vec ( X ) (cid:107) 1 ) .
Revised: In this way, we now can input the regularized afﬁnity matrix K reg to the GNN, to better learn the impact of the regularization term f ( (cid:107) vec ( X ) (cid:107) 1 ) .

Original: Fig.
Revised: Fig.

Original: 3 shows the results of the regularized afﬁnity matrix.
Revised: 3 shows the results of the regularized afﬁnity matrix.

Original: Notably, some values in the regularized afﬁnity K reg become negative.
Revised: Notably, some values in the regularized afﬁnity K reg become negative.

Original: Intuitively, the negative elements in the afﬁnity matrix denote that the afﬁnity score maximization reward may no longer pursue to match as many keypoints pairs as possible, which prevents the agent from picking up outliers to some extent.
Revised: Intuitively, the negative elements in the afﬁnity matrix denote that the afﬁnity score maximization reward may no longer pursue to match as many keypoints pairs as possible, which prevents the agent from picking up outliers to some extent.

Original: Besides, most traditional graph matching solvers (Cho et al., 2010; Egozi et al., 2013) assume the afﬁnity matrix is non-negative, while our RGM has no such restriction.
Revised: Besides, most traditional graph matching solvers (Cho et al., 2010; Egozi et al., 2013) assume the afﬁnity matrix is non-negative, while our RGM has no such restriction.

Original: Fig.
Revised: Fig.

Original: 4 illustrates the effectiveness of regularized afﬁnity score , with three different regularization functions { f i ( n ) } i =1 , 2 , 3 .
Revised: 4 illustrates the effectiveness of regularized afﬁnity score , with three different regularization functions { f i ( n ) } i =1 , 2 , 3 .

Original: We construct a set of permutation solutions by GM solver, and calculate the F1 score and afﬁnity score of the original afﬁnity matrix and regularized afﬁnity matrix.
Revised: We construct a set of permutation solutions by GM solver, and calculate the F1 score and afﬁnity score of the original afﬁnity matrix and regularized afﬁnity matrix.

Original: As one can see, the afﬁnity score of the original afﬁnity matrix ﬂuctuates a lot with the increase of the F score.
Revised: As one can see, the afﬁnity score of the original afﬁnity matrix ﬂuctuates a lot with the increase of the F score.

Original: Besides, there are some cases where original afﬁnity is larger than ground truth matching.
Revised: Besides, there are some cases where original afﬁnity is larger than ground truth matching.

Original: In contrast, the regularized afﬁnity score of the regularized afﬁnity matrix is more stable and overall nearly positively correlates to the F1 score.
Revised: In contrast, the regularized afﬁnity score of the regularized afﬁnity matrix is more stable and overall nearly positively correlates to the F1 score.

Original: It proves that the regularized afﬁnity score is a better optimization objective that is consistent with matching accuracy (F1 score).
Revised: It proves that the regularized afﬁnity score is a better optimization objective that is consistent with matching accuracy (F1 score).

Original: This approximate optimization process can be merged into our revocable framework as shown in
Revised: This approximate optimization process can be merged into our revocable framework as shown in

Original: Alg. 1.
Revised: Alg. 1.

Original: In the sequential decision making process, at every step when the agent needs to select the next vertex, we make an approximate optimization based on the current solution, and get the regularized afﬁnity matrix K reg .
Revised: In the sequential decision making process, at every step when the agent needs to select the next vertex, we make an approximate optimization based on the current solution, and get the regularized afﬁnity matrix K reg .

Original: Then, we feed the regularized afﬁnity matrix K reg instead of the original afﬁnity matrix K to the GNN.
Revised: Then, we feed the regularized afﬁnity matrix K reg instead of the original afﬁnity matrix K to the GNN.

Original: The above introduced regularization technique can be readily adopted in our sequential node matching scheme.
Revised: The above introduced regularization technique can be readily adopted in our sequential node matching scheme.

Original: In contrast, it is nontrivial for them to be integrated with existing deep GM methods which are mainly performed in one shot for the whole matching.[Table] [Table]
Revised: In contrast, it is nontrivial for them to be integrated with existing deep GM methods which are mainly performed in one shot for the whole matching.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We perform experiments on the standard benchmarks including image data (synthetic image,Willow
Revised: We perform experiments on the standard benchmarks including image datasets (synthetic image,

Original:  Object, Pascal VOC) as well as pure combinatorial optimization instances (QAPLIB), as  the latter isespecially suited for back-end solver.
Revised: Willow Object, Pascal VOC) as well as pure combinatorial optimization instances (QAPLIB), as [Table] [Table] the latter is suited for back-end solver.

Original: Due to the page limit, the detailed experiment protocols are introduced in the appendix (A.3), including hyperparameter settings (A.3.1), evaluation metrics (A.3.2), compared methods (A.3.3), dataset settings (A.3.4), and the train/test protocols (A.3.5).
Revised: Due to the page limit, the detailed experiment protocols are introduced in the appendix (A.3), including hyperparameter settings (A.3.1), evaluation metrics (A.3.2), compared methods (A.3.3), dataset settings (A.3.4), and the train/test protocols (A.3.5).

Original: 5. E XPERIMENTS ON W ILLOW O BJECT D ATASET WITH O UTLIERS
Revised: 5. E XPERIMENTS ON W ILLOW O BJECT D ATASET WITH O UTLIERS

Original: 1) Performance over Different Amounts of Outliers.
Revised: 1) Performance over Different Amounts of Outliers.

Original: We conduct more experiments with respect to different amounts of outliers.
Revised: We conduct more experiments with respect to different amounts of outliers.

Original: In these experiments, the number of inliers is ﬁxed at 10 while the number of outliers varies from 1 to 6.
Revised: In these experiments, the number of inliers is ﬁxed at 10 while the number of outliers varies from 1 to 6.

Original: The experiment setup follows Jiang et al.
Revised: The experiment setup follows Jiang et al.

Original: (2021b), and the evaluation is performed on all ﬁve categories.
Revised: (2021), and the evaluation is performed on all ﬁve categories.

Original: For the training and testing process, we follow the data split rate in BBGM (Rol ´ ınek et al., 2020) in all experiments.
Revised: For the training and testing process, we follow the data split rate in BBGM (Rol ´ ınek et al., 2020) in all experiments.

Original: Table 1 shows the performance of our method and baselines in the average of all ﬁve classes on the Willow Object dataset with respect to different amounts of outliers (from 1 to 6).
Revised: Table 1 shows the performance of our method and baselines in the average of all ﬁve classes on the Willow Object dataset with respect to different amounts of outliers (from 1 to 6).

Original: For our methods at the bottom, “RGM + AR” denotes
Revised: For our methods at the bottom, “RGM + AR” denotes

Original: RGM with afﬁnity regularization, “RGM + IC” denotes RGM with inlier count information, and “RGM + AR + IC” denotes RGM with both.
Revised: RGM with afﬁnity regularization, “RGM + IC” denotes RGM with inlier count information, and “RGM + AR + IC” denotes RGM with both.

Original: For ablation, RGM without the revocable mechanism is shown in the last three columns, denoted by “w/o rev.”.
Revised: For ablation, RGM without the revocable mechanism is shown in the last three columns, denoted by “w/o rev.”.

Original: Regularized afﬁnity and inlier count information are two ways to handle the outliers introduced in Sec. 4.3.
Revised: Regularized afﬁnity and inlier count information are two ways to handle the outliers introduced in Sec. 4.3.

Original: RGM reaches the best results over all ﬁve classes in this dataset, and gets a 4% improvement in average compared to the best baseline BBGM.
Revised: RGM reaches the best results over all ﬁve classes in this dataset, and gets a 4% improvement in average compared to the best baseline BBGM.

Original: For the variants of our methods, “RGM + AR + IC” shows the best performance, while “RGM + AR” requires no extra information and still outperforms all the baselines.
Revised: For the variants of our methods, “RGM + AR + IC” shows the best performance, while “RGM + AR” requires no extra information and still outperforms all the baselines.

Original: 2) Sensitivity to the Input of Inlier Count.
Revised: 2) Sensitivity to the Input of Inlier Count.

Original: Since in reality, the inlier count information might not be always accurate, we test our methods’ robustness against the inexact number of inliers.
Revised: Since in reality, the inlier count information might not be always accurate, we test our methods’ robustness against the inexact number of inliers.

Original: We reconduct experiments, of which each image contains 10 inliers and 3 outliers.
Revised: We reconduct experiments, of which each image contains 10 inliers and 3 outliers.

Original: We modify the inlier count info n i to (8 - 13) instead of ground truth 10, as the input to “RGM + IC” and “RGM + AR + IC”.
Revised: We modify the inlier count info n i to (8 - 13) instead of ground truth 10, as the input to “RGM + IC” and “RGM + AR + IC”.

Original: The results in Table 2 show that the performances of these two methods tend to downgrade given incorrect information, as expected, but they can still outperform BBGM in cases when the information is only a little biased.
Revised: The results in Table 2 show that the performances of these two methods tend to downgrade given incorrect information, as expected, but they can still outperform BBGM in cases when the information is only a little biased.

Original: Meanwhile, the regularized afﬁnity matrix can also enhance the robustness.
Revised: Meanwhile, the regularized afﬁnity matrix can also enhance the robustness.

Original: As a result, our RGM can still work well even the inlier count information is inaccurate.
Revised: As a result, our RGM can still work well even the inlier count information is inaccurate.

Original: 5. E XPERIMENTS ON P ASCAL VOC
Revised: 5. E XPERIMENTS ON P ASCAL VOC

Original: Table 3 reports the results on the Pascal VOC.
Revised: Table 3 reports the results on the Pascal VOC.

Original: We also apply the train/test split rate as mentioned in Sec.
Revised: We also apply the train/test split rate as mentioned in Sec.

Original: A.3.5.
Revised: A.3.5.

Original: We show the average performance of every 20 classes in this dataset, and we can see that RGM outperforms all baselines on Pascal VOC.
Revised: We show the average performance of every 20 classes in this dataset, and we can see that RGM outperforms all baselines on Pascal VOC.

Original: Speciﬁcally, the baselines on the left side are
Revised: Speciﬁcally, the baselines on the left side are

Original: [Table] [Table]
Revised: [Table] [Table]

Original: label-free methods, which use the same input as RGM thus leading to a fair comparison.
Revised: label-free methods, which use the same input as RGM thus leading to a fair comparison.

Original: In other words, the left side methods are the pure back-end solvers that pursue the highest objective score.
Revised: In other words, the left side methods are the pure back-end solvers that pursue the highest objective score.

Original: We can see that our RGM reaches the highest objective score 1.040, which is clearly greater than 1.
Revised: We can see that our RGM reaches the highest objective score 1.040, which is clearly greater than 1.

Original: However, even with this high objective score, the matching accuracy of RGM is still unsatisfactory.
Revised: However, even with this high objective score, the matching accuracy of RGM is still unsatisfactory.

Original: Then, we conduct experiments with methods on the right side that requires label as supervision, which means their backend-solvers are trained to reach the higher accuracy instead of the objective score.
Revised: Then, we conduct experiments with methods on the right side that requires label as supervision, which means their backend-solvers are trained to reach the higher accuracy instead of the objective score.

Original: We combine our RGM with SOTA method NGM and NGM-v2, by using their well-trained model as the guidance of RGM for imitation learning.
Revised: We combine our RGM with SOTA method NGM and NGM-v2, by using their well-trained model as the guidance of RGM for imitation learning.

Original: Then, we can ﬁnd that RGM can boost the performance of original methods with the modiﬁed direction, which expands the usage of our RGM.
Revised: Then, we can ﬁnd that RGM can boost the performance of original methods with the modiﬁed direction, which expands the usage of our RGM.

Original: 5. E XPERIMENTS ON QAPLIB D ATASET
Revised: 5. E XPERIMENTS ON QAPLIB D ATASET

Original: For QAPLIB (Burkard et al., 1997), we use the settings exactly the same as NGM.
Revised: For QAPLIB (Burkard et al., 1997), we use the settings exactly the same as NGM.

Original: The results are shown in Table 4, where “esc(16-64)” denotes that the size of class “esc” varies from 16 to 64.
Revised: The results are shown in Table 4, where “esc(16-64)” denotes that the size of class “esc” varies from 16 to 64.

Original: The train-test split rate for this dataset is (1 : 1).
Revised: The train-test split rate for this dataset is (1 : 1).

Original: We calculate the gap between the computed solution and the optimal, and report the average optimal gap (the lower the better).
Revised: We calculate the gap between the computed solution and the optimal, and report the average optimal gap (the lower the better).

Original: Besides, inference time per instance is listed in the last column.
Revised: Besides, inference time per instance is listed in the last column.

Original: We compare with four existing solvers: SM (Leordeanu &
Revised: We compare with four existing solvers: SM (Leordeanu &

Original: Hebert, 2005), Sinkhorn-JA (Kushinsky et al., 2019), RRWM (Cho et al., 2010) and NGM (Wang et al., 2021a).
Revised: Hebert, 2005), Sinkhorn-JA (Kushinsky et al., 2019), RRWM (Cho et al., 2010) and NGM (Wang et al., 2021a).

Original: Note that the NGM is the ﬁrst that utilizes deep learning to solve QAP which is an emerging topic.
Revised: Note that the NGM is the ﬁrst that utilizes deep learning to solve QAP which is an emerging topic.

Original: It shows that RGM outperforms all the baselines, including the latest solver NGM.
Revised: It shows that RGM outperforms all the baselines, including the latest solver NGM.

Original: 5. A DDITIONAL E XPERIMENTS
Revised: 5. A DDITIONAL E XPERIMENTS

Original: Due to the page limit, we have to move many important results, tables, and ﬁgures into the appendix (A.4).

Original: Here we list an index of those experiments in the appendix.
Revised: Here we list an index of those experiments in the appendix.

Original: If you are interested in several experiments listed below, please refer to the appendix.
Revised: If you are interested in several experiments listed below, please refer to Appendix.

Original: (A.4.1)
Revised: (A.4.1)

Original: Experiments on the synthetic dataset : evaluating RGM on the synthetic images.
Revised: Experiments on the synthetic dataset : evaluating RGM on the synthetic images.

Original: (A.4.2) Experiments on the different categories of the
Revised: (A.4.2) Experiments on the different categories of the

Original: Willow Object : results over ﬁve categories on the Willow Object dataset.
Revised: Willow Object : results over ﬁve categories on the Willow Object dataset.

Original: (A.4.3) Generalization test of RGM : generalization to different amounts of outliers, similar categories, and different classes of QAP instances.
Revised: (A.4.3) Generalization test of RGM : generalization to different amounts of outliers, similar categories, and different classes of QAP instances.

Original: (A.4.4) Hyperparameter sensitivity study : testing the sensitivity of the hyperparameters in RGM.
Revised: (A.4.4) Hyperparameter sensitivity study : testing the sensitivity of the hyperparameters in RGM.

Original: (A.4.5)
Revised: (A.4.5)

Original: Seeded graph matching : graph matching with several initial seeds.
Revised: Seeded graph matching : graph matching with several initial seeds.

Original: (A.4.6) Revocable action v.s. Local Rewrite : comparison of RGM and popular Local Rewrite framework (Chen & Tian, 2019).
Revised: (A.4.6) Revocable action v.s. Local Rewrite : comparison of RGM and popular Local Rewrite framework (Chen & Tian, 2019).

Original: (A.4.7) Alternative RL backbones : testing alternative RL backbones instead of D3QN in RGM.
Revised: (A.4.7) Alternative RL backbones : testing alternative RL backbones instead of D3QN in RGM.

Original: (A.4.8) Limitation analysis of the inconsistency : analysis of the inconsistency between the afﬁnity score and the F1 score in the aforementioned experiments.
Revised: (A.4.8) Limitation analysis of the inconsistency : analysis of the inconsistency between the afﬁnity score and the F1 score in the aforementioned experiments.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have presented a deep RL based approach for graph matching, especially in the presence of outliers.
Revised: We have presented a deep RL based approach for graph matching, especially in the presence of outliers.

Original: The sequential decision scheme allows to natural select the inliers for matching and avoids matching outliers.
Revised: The sequential decision scheme allows to natural select the inliers for matching and avoids matching outliers.

Original: To our best knowledge, it is the ﬁrst work for RL of graph matching which can be applied to its general QAP form.
Revised: To our best knowledge, it is the ﬁrst work for RL of graph matching which can be applied to its general QAP form.

Original: We further devise two techniques to improve the robustness.
Revised: We further devise two techniques to improve the robustness.

Original: The ﬁrst is the revocable action mechanism which is shown well suited to our complex constrained search procedure.
Revised: The ﬁrst is the revocable action mechanism which is shown well suited to our complex constrained search procedure.

Original: The other is the afﬁnity regularization based on parametric function ﬁtting, which is shown can effectively refrain the agent from matching outliers when the number of inliers is unknown.
Revised: The other is the afﬁnity regularization based on parametric function ﬁtting, which is shown can effectively refrain the agent from matching outliers when the number of inliers is unknown.

Original: Experiments on multiple real-world datasets show the cost-effectiveness of RGM.
Revised: Experiments on multiple real-world datasets show the cost-effectiveness of RGM.

Original: A A PPENDIX A. D ISCUSSION WITH EXISTING WORK
Revised: A A PPENDIX A. D ISCUSSION WITH EXISTING WORK

Original: Instead of the Lawler’s QAP, several GM works choose the Koopmans-Beckmann’s QAP (Koopmans & Beckmann, 1957), which requires the explicit input of two graphs.
Revised: Instead of the Lawler’s QAP, several GM works choose the Koopmans-Beckmann’s QAP (Koopmans & Beckmann, 1957), which requires the explicit input of two graphs.

Original: We argue that such raw information may not always be available in practice e.g. for privacy.
Revised: We argue that such raw information may not always be available in practice e.g. for privacy.

Original: For graph matching, the most general form is Lawler’s QAP (Lawler, 1963) whose input is the pairwise afﬁnity matrix whereby the raw graph information is removed and the Koopmans-Beckmann’s QAP is a special case for
Revised: For graph matching, the most general form is Lawler’s QAP (Lawler, 1963) whose input is the pairwise afﬁnity matrix whereby the raw graph information is removed and the Koopmans-Beckmann’s QAP is a special case for

Original: Lawler’s QAP.
Revised: Lawler’s QAP.

Original: There are also standing and widely adopted public benchmarks for Lawler’s QAP e.g. QAPLIB (Burkard et al., 1997).
Revised: There are also standing and widely adopted public benchmarks for Lawler’s QAP e.g. QAPLIB (Burkard et al., 1997).

Original: For its generality and popularity, SOTA deep GM works (Wang et al., 2020c; 2021a; Rol ´ ınek et al., 2020) follow this line, whereby a GNN model is applied on the so-called association graph whose weighted adjacency matrix is the afﬁnity matrix.
Revised: For its generality and popularity, SOTA deep GM works (Wang et al., 2020c; 2021a; Rol ´ ınek et al., 2020) follow this line, whereby a GNN model is applied on the so-called association graph whose weighted adjacency matrix is the afﬁnity matrix.

Original: This GNN model is trained for node embedding on the association graph, which selects the node correspondence via supervised learning in one shot.
Revised: This GNN model is trained for node embedding on the association graph, which selects the node correspondence via supervised learning in one shot.

Original: Since Lawler’s QAP only requires the afﬁnity matrix and keeps the node and edge information of user unknown, the privacy can be better retained than explicitly using the input graph as done in the Koopmans-Beckmann’s QAP.
Revised: Since Lawler’s QAP only requires the afﬁnity matrix and keeps the node and edge information of user unknown, the privacy can be better retained than explicitly using the input graph as done in the Koopmans-Beckmann’s QAP.

Original: Therefore, Lawler’s
Revised: Therefore, Lawler’s

Original: QAP has been adapted to several privacy-sensitive tasks, including model fusion (Liu et al., 2022), bioinformatics (Zaslavskiy et al., 2009), and text alignment (Fey et al., 2020b).
Revised: QAP has been adapted to several privacy-sensitive tasks, including model fusion (Liu et al., 2022), bioinformatics (Zaslavskiy et al., 2009), and text alignment (Fey et al., 2020b).

Original: As mentioned in the paper, utilizing label-free revocable RL with afﬁnity regularization for designing a new back-end solver becomes a promising tool for pushing the frontier of graph matching research.
Revised: As mentioned in the paper, utilizing label-free revocable RL with afﬁnity regularization for designing a new back-end solver becomes a promising tool for pushing the frontier of graph matching research.

Original: However, here we emphasize our method is focused on the back-end part whose input is the afﬁnity matrix, and our method cannot be combined with learnable CNN and GNN for input graph feature extraction and metric learning part, for joint differentiable front-back-end learning.
Revised: However, here we emphasize our method is focused on the back-end part whose input is the afﬁnity matrix, and our method cannot be combined with learnable CNN and GNN for input graph feature extraction and metric learning part, for joint differentiable front-back-end learning.

Original: The reason is that, as will be shown in our approach, the RL reward isa function parameterized by the front-end CNN/GNN/MLP, which makes end-to-end impossible.
Revised: The reason is that, as will be shown in our approach, the RL reward is parameterized by the frontend CNN/GNN/MLP, which makes end-to-end impossible.

Original: For the above reason, our RL solver is trained on the input afﬁnity matrix, by ﬁxing the parameters of the front-end models which can be pretrained via existing supervised methods (Wang et al., 2019b).
Revised: For the above reason, our RL solver is trained on the input afﬁnity matrix, by ﬁxing the parameters of the front-end models which can be pretrained via existing supervised methods (Wang et al., 2019b).

Original: Table 5 compares existing works for their learning modules and techniques.
Revised: Table 5 compares existing works for their learning modules and techniques.

Original: This protocol is akin to the QAP learning part in (Wang et al., 2021a), which can also be regarded as the inherent limitation with RL.
Revised: This protocol is akin to the QAP learning part in (Wang et al., 2021a), which can also be regarded as the inherent limitation with RL.

Original: Fortunately, as will be shown in ourextensive experiments, our method still can outperform end-to-end supervised methods, especially given a large ratio of outliers.
Revised: Fortunately, as will be shown in our experiments, our method still can outperform end-to-end supervised methods, especially with a large ratio of outliers.

Original: Our two-stage training pipeline is also more efﬁcient than joint learning, and thus suitable for our method as RL is more costly than supervised learning. 
Revised: Our two-stage training pipeline is also more efﬁcient than joint learning, and thus suitable for our method as RL is more costly than supervised learning. [Equation]

Original: A. D ETAILS OF THE PROPOSED METHOD
Revised: A. D ETAILS OF THE PROPOSED METHOD

Original: The idea of RL is to learn from the interactions between the agent and the environment (Sutton & Barto, 2005).
Revised: The idea of RL is to learn from the interactions between the agent and the environment (Sutton & Barto, 2005).

Original: The agent’s observation of the current environment is called state s .
Revised: The agent’s observation of the current environment is called state s .

Original: The agent chooses an action a given the current state by a speciﬁc policy.
Revised: The agent chooses an action a given the current state by a speciﬁc policy.

Original: After the agent performs an action, the environment will transfer to another state s (cid:48) .
Revised: After the agent performs an action, the environment will transfer to another state s (cid:48) .

Original: Meanwhile, the environment will feedback to the agent with a reward r .
Revised: Meanwhile, the environment will feedback to the agent with a reward r .

Original: This pipeline solves the problem progressively.
Revised: This pipeline solves the problem progressively.

Original: For graph matching, “progressively” means to select the vertex in the association graph one by one.
Revised: For graph matching, “progressively” means to select the vertex in the association graph one by one.

Original: The environment is deﬁned as a partial solution to the original combinatorial problem (Eq. 1) and equivalently the association graph, where the reward denotes the improvement of the objective function by matching a new pair of nodes.
Revised: The environment is deﬁned as a partial solution to the original combinatorial problem (Eq. 1) and equivalently the association graph, where the reward denotes the improvement of the objective function by matching a new pair of nodes.

Original: The interactions between the agent and the environment are recorded as a transition ( s, a, r, s (cid:48) ) into an experience replay memory M .
Revised: The interactions between the agent and the environment are recorded as a transition ( s, a, r, s (cid:48) ) into an experience replay memory M .

Original: After several episodes, the agent updates its networks f
Revised: After several episodes, the agent updates its networks f

Original: θ according to the transitions sampled from M .
Revised: θ according to the transitions sampled from M .

Original: A.2. N ETWORK S TRUCTURE
Revised: A.2. N ETWORK S TRUCTURE

Original: 1) State Representation Networks.
Revised: 1) State Representation Networks.

Original: To better represent the current state on the association graph, we choose graph neural networks (GNN) (Kipf & Welling, 2017) to compute its embedding.
Revised: To better represent the current state on the association graph, we choose graph neural networks (GNN) (Kipf & Welling, 2017) to compute its embedding.

Original: GNN extracts the vertex features based on their adjacent neighbors.
Revised: GNN extracts the vertex features based on their adjacent neighbors.

Original: To better use the edge weights in the association graph, we derive from the idea of struct2vec (Dai et al., 2016).
Revised: To better use the edge weights in the association graph, we derive from the idea of struct2vec (Dai et al., 2016).

Original: In our embedding networks, the current solution, node weights, and edge weights of the association graph are considered.
Revised: In our embedding networks, the current solution, node weights, and edge weights of the association graph are considered.

Original: The embedding formula is:
Revised: The embedding formula is:

Original: [Equation]
Revised: [Equation]

Original: where E t ∈ R n 1 n 2 × d denotes the embedding in the t -th iteration, with d as the hidden size.
Revised: where E t ∈ R n 1 n 2 × d denotes the embedding in the t -th iteration, with d as the hidden size.

Original: At every iteration, the embedding is calculated by four hidden parts
Revised: At every iteration, the embedding is calculated by four hidden parts

Original: h 1 , h 2 , h 3 , h 4 ∈ R n 1 n 2 × d .
Revised: h 1 , h 2 , h 3 , h 4 ∈ R n 1 n 2 × d .

Original: θ 1 ∈ R d , θ
Revised: θ 1 ∈ R d , θ

Original: 2 ∈ R d × d , θ 3 ∈ R d , θ 4 ∈ R d × d and θ 5 ∈ R
Revised: 2 ∈ R d × d , θ 3 ∈ R d , θ 4 ∈ R d × d and θ 5 ∈ R

Original: d are the weight matrices in the neural networks.
Revised: d are the weight matrices in the neural networks.

Original: t is the index of the iteration and the total number of the iterations is T .
Revised: t is the index of the iteration and the total number of the iterations is T .

Original: We set the initial embedding
Revised: We set the initial embedding

Original: E 0 = 0 and use ReLU as the activation function.
Revised: E 0 = 0 and use ReLU as the activation function.

Original: As for the hidden parts, each hidden part represents a kind of feature: h 1 is to calculate the impact of current permutation matrix X (cid:48) which is transformed from the current partial solution U (cid:48) .
Revised: As for the hidden parts, each hidden part represents a kind of feature: h 1 is to calculate the impact of current permutation matrix X (cid:48) which is transformed from the current partial solution U (cid:48) .

Original: h 2 is to take neighbor’s embedding into consideration, where A is the adjacency matrix of the association graph and divide ( n 1 − 1)( n 2 − 1) is for average since every vertex has ( n 1 − 1)( n 2 − 1) neighbors.
Revised: h 2 is to take neighbor’s embedding into consideration, where A is the adjacency matrix of the association graph and divide ( n 1 − 1)( n 2 − 1) is for average since every vertex has ( n 1 − 1)( n 2 − 1) neighbors.

Original: h 3 calculates the average of neighbor’s vertex weights, where F is the vertex weight matrix.
Revised: h 3 calculates the average of neighbor’s vertex weights, where F is the vertex weight matrix.

Original: h 4 is designed to extract the features of adjacent edges, where W is the edge weight matrix.
Revised: h 4 is designed to extract the features of adjacent edges, where W is the edge weight matrix.

Original: Please note that the core inputs of our GNN are the permutation matrix X and the afﬁnity matrix K ( A , F and
Revised: Please note that the core inputs of our GNN are the permutation matrix X and the afﬁnity matrix K ( A , F and

Original: W are derived from the afﬁnity matrix K ).
Revised: W are derived from the afﬁnity matrix K ).

Original: 2) Q-Value Estimation Networks.
Revised: 2) Q-Value Estimation Networks.

Original: The Q-learning based algorithms use Q ( s, a ) to represent the value of taking action a in state s , as an expected value of the acquired reward after choosing this action.
Revised: The Q-learning based algorithms use Q ( s, a ) to represent the value of taking action a in state s , as an expected value of the acquired reward after choosing this action.

Original: The agent picks the next action given the estimation of Q ( s, a ) .
Revised: The agent picks the next action given the estimation of Q ( s, a ) .

Original: The Q-value estimation network f θ takes the embedding of the current state as input and predicts the Q-value for each possible action.
Revised: The Q-value estimation network f θ takes the embedding of the current state as input and predicts the Q-value for each possible action.

Original: We adopt Dueling DQN (Wang et al., 2016) as our approximator to estimate the
Revised: We adopt Dueling DQN (Wang et al., 2016) as our approximator to estimate the

Original: Q-value function.
Revised: Q-value function.

Original: The architecture of our f θ is:
Revised: The architecture of our f θ is:

Original: [Equation]
Revised: [Equation]

Original: where E (cid:62) is the ﬁnal output of the embedding network by Eq.
Revised: where E (cid:62) is the ﬁnal output of the embedding network by Eq.

Original: h 5 ∈ R n 1 n 2 × d is the hidden layer for embedding.
Revised: h 5 ∈ R n 1 n 2 × d is the hidden layer for embedding.

Original: h v ∈ R 1 is the hidden layer for the state function.
Revised: h v ∈ R 1 is the hidden layer for the state function.

Original: h a ∈ R n 1 n 2 is the hidden layer [Equation] else then Calculate then
Revised: h a ∈ R n 1 n 2 is the hidden layer for the advantage function.

Original: ; if (Inlier Count and | s |

Original: == n i ) or (current best solution unchanged in T es rounds) then break ; for the advantage function.

Original: θ 6 ∈ R d × d , θ 7 ∈ R d , θ 8 ∈ R
Revised: θ 6 ∈ R d × d , θ 7 ∈ R d , θ 8 ∈ R

Original: d are the weights of the neural networks.
Revised: d are the weights of the neural networks.

Original: b  and b 3 are  is  output  our Q-value estimate network.
Revised: b 1 , b 2 , and b 3 are the bias vectors. Q ∈ R n 1 n 2 is the ﬁnal output of our Q-value estimate network.

Original: It predicts the value of each action given the current state.
Revised: It predicts the value of each action given the current state.

Original: The state function and advantage function are designed to separate the value of state and action.
Revised: The state function and advantage function are designed to separate the value of state and action.

Original: Speciﬁcally, the state function predicts the value of different states and the advantage predicts the value of each action given the particular state.
Revised: Speciﬁcally, the state function predicts the value of different states and the advantage predicts the value of each action given the particular state.

Original: The previous work (Wang et al., 2016) shows that this dueling architecture can better learn the impact of different actions.
Revised: The previous work (Wang et al., 2016) shows that this dueling architecture can better learn the impact of different actions.

Original: Besides, we force the sum of the output vector of the advantage function to 0 by subtracting the mean of it, which makes the separation of the state value and advantage easier.
Revised: Besides, we force the sum of the output vector of the advantage function to 0 by subtracting the mean of it, which makes the separation of the state value and advantage easier.

Original: We use Q ( s, a ; f θ ) to denote the estimated
Revised: We use Q ( s, a ; f θ ) to denote the estimated

Original: Q-value by f θ when the agent takes action a on state s .
Revised: Q-value by f θ when the agent takes action a on state s .

Original: A.2. E XPERIENCE R EPLAY M EMORY
Revised: A.2. E XPERIENCE R EPLAY M EMORY

Original: For sample efﬁciency, we maintain a prioritized experience replay memory M (Schaul et al., 2016) that stores the experience of the agent, deﬁned as the transition ( s i , a i , r i , s (cid:48) i ) (denoting state, action, reward, and state of next step respectively).
Revised: For sample efﬁciency, we maintain a prioritized experience replay memory M (Schaul et al., 2016) that stores the experience of the agent, deﬁned as the transition ( s i , a i , r i , s (cid:48) i ) (denoting state, action, reward, and state of next step respectively).

Original: As the training progresses, we add new transitions to
Revised: As the training progresses, we add new transitions to

Original: M and remove old transitions.
Revised: M and remove old transitions.

Original: The agents will take samples from the experience replay memory to update their neural networks.
Revised: The agents will take samples from the experience replay memory to update their neural networks.

Original: We follow the idea of prioritized experience replay memory, which adds a priority for each transition and higher priority denotes higher probabilities to be sampled:
Revised: We follow the idea of prioritized experience replay memory, which adds a priority for each transition and higher priority denotes higher probabilities to be sampled:

Original: [Equation]
Revised: [Equation]

Original: (11) where P ( i ) is the probability and p i is the priority of the i -th transition.
Revised: where P ( i ) is the probability and p i is the priority of the i -th transition.

Original: α is a hyperparameter.
Revised: α is a hyperparameter.

Original: The calculation of p i is based on the underﬁtting extent of the transition, and a larger bias rate of the agent’s Q-value estimation means a relatively higher priority.
Revised: The calculation of p i is based on the underﬁtting extent of the transition, and a larger bias rate of the agent’s Q-value estimation means a relatively higher priority.

Original: A.2. M ODEL U PDATING
Revised: A.2. M ODEL U PDATING

Original: We follow Double DQN (Hasselt et al., 2016) to calculate the loss function and update the parameters.
Revised: We follow Double DQN (Hasselt et al., 2016) to calculate the loss function and update the parameters.

Original: We pick the next action a (cid:48) by the current Q-value estimate network f θ , but use the target
Revised: We pick the next action a (cid:48) by the current Q-value estimate network f θ , but use the target

Original: Q-value estimate network f θ − to predict its value as Eq.
Revised: Q-value estimate network f θ − to predict its value as Eq.

Original: 12 shows.
Revised: 12 shows.

Original: The motivation of designing this loss function is: the Q-value that is overestimated in one network will be mitigated to an extent in another network.
Revised: The motivation of designing this loss function is: the Q-value that is overestimated in one network will be mitigated to an extent in another network.

Original: [Equation]
Revised: [Equation]

Original: where L ( · ) is the loss function to be optimized.
Revised: where L ( · ) is the loss function to be optimized.

Original: f θ is the current Q-value estimation network and f θ − is the target Q-value estimation network.
Revised: f θ is the current Q-value estimation network and f θ − is the target Q-value estimation network.

Original: s, a, r, s (cid:48) , a (cid:48) stand for the state, action, reward, next state, and next action.
Revised: s, a, r, s (cid:48) , a (cid:48) stand for the state, action, reward, next state, and next action.

Original: The target network f θ − shares the same architecture with f θ and the parameters of f θ − will be replaced by the parameters of f θ every period.
Revised: The target network f θ − shares the same architecture with f θ and the parameters of f θ − will be replaced by the parameters of f θ every period.

Original: The design of such a target network is for keeping the target Q value remains unchanged for a period of time, which reduces the correlation between the current Q value and the target one and improves the training stability.
Revised: The design of such a target network is for keeping the target Q value remains unchanged for a period of time, which reduces the correlation between the current Q value and the target one and improves the training stability.

Original: A.2. F URTHER D ISCUSSION OF THE R EVOCABLE A CTION F RAMEWORK
Revised: A.2. F URTHER D ISCUSSION OF THE R EVOCABLE A CTION F RAMEWORK

Original: Note that our revocable action mechanism requires most changes in our environment settings.
Revised: Note that our revocable action mechanism requires most changes in our environment settings.

Original: Therefore, we design a new RL environment for the revocable action mechanism, and the main differences from the original environment are: the available set is gone; the agent can choose any vertex in the vertices
Revised: Therefore, we design a new RL environment for the revocable action mechanism, and the main differences from the original environment are: the available set is gone; the agent can choose any vertex in the vertices

Original: V a of the association graph G a ; when the environment receives a vertex that is in conﬂict with the current partial solution, it releases the conﬂicted vertices and adds the new-coming vertex to the partial solution.
Revised: V a of the association graph G a ; when the environment receives a vertex that is in conﬂict with the current partial solution, it releases the conﬂicted vertices and adds the new-coming vertex to the partial solution.

Original: The agent design is irrelevant to the basic or the revocable environment, and the training process of the revocable framework almost remains the same, as Alg.
Revised: The agent design is irrelevant to the basic or the revocable environment, and the training process of the revocable framework almost remains the same, as Alg.

Original: 1 shows.
Revised: 1 shows.

Original: While the revocable ﬂexibility also brings some side effects, e.g. we can not adopt the acceleration tricks for
Revised: While the revocable ﬂexibility also brings some side effects, e.g. we can not adopt the acceleration tricks for

Original: GNN, such as dynamic embedding (Wang et al., 2021b), which can otherwise speedup the inference time.
Revised: GNN, such as dynamic embedding (Wang et al., 2021b), which can otherwise speedup the inference time.

Original: As for the stopping rules for our revocable action mechanism, there are three stopping cases and the current best solution will be returned: 1) when the number of matched nodes equals to the given or estimated input inlier count; 2) no afﬁnity score improvement is made (the current best solution remains the same) in T es rounds where T es is the given hyperparameter for early stopping; 3) the number of rounds reaches the preset max round parameter T max .
Revised: As for the stopping rules for our revocable action mechanism, there are three stopping cases and the current best solution will be returned: 1) when the number of matched nodes equals to the given or estimated input inlier count; 2) no afﬁnity score improvement is made (the current best solution remains the same) in T es rounds where T es is the given hyperparameter for early stopping; 3) the number of rounds reaches the preset max round parameter T max .

Original: Besides, we add a small reward penalty r p in each step to make the agent away from repeatedly taking an action and canceling it.
Revised: Besides, we add a small reward penalty r p in each step to make the agent away from repeatedly taking an action and canceling it.

Original: The usage of the these parameters can be seen in Alg. 1.
Revised: The usage of the these parameters can be seen in Alg. 1.

Original: To our best knowledge, there are in general two existing techniques allowing revocable actions, at least for RL-based combinatorial optimization: Local Rewrite (Chen & Tian, 2019) and ECODQN (Barrett et al., 2020).
Revised: To our best knowledge, there are in general two existing techniques allowing revocable actions, at least for RL-based combinatorial optimization: Local Rewrite (Chen & Tian, 2019) and ECODQN (Barrett et al., 2020).

Original: Here we discuss our difference from these methods.
Revised: Here we discuss our difference from these methods.

Original: The local rewrite framework keeps improving the solution given as input by exchange the parts of it.
Revised: The local rewrite framework keeps improving the solution given as input by exchange the parts of it.

Original: However, the performance of the local rewrite highly relies on the input solution.
Revised: However, the performance of the local rewrite highly relies on the input solution.

Original: In our empirical tries, the efﬁciency and effectiveness of local rewrite are unsatisfactory, as will be shown in our experiments.
Revised: In our empirical tries, the efﬁciency and effectiveness of local rewrite are unsatisfactory, as will be shown in our experiments.

Original: The ECO-DQN framework does have a promising performance in the Maximum
Revised: The ECO-DQN framework does have a promising performance in the Maximum

Original: Cut problem, but it is inherently designed to work on this speciﬁc Maximum Cut problem, which only has fewer or no constraints, and is clearly impossible to adapt to the graph matching problem.
Revised: Cut problem, but it is inherently designed to work on this speciﬁc Maximum Cut problem, which only has fewer or no constraints, and is clearly impossible to adapt to the graph matching problem.

Original: Therefore, in this paper, we devise a new revocable framework RGM to meet the characteristic of the graph matching problem, which is more friendly to graph with relatively more constraints.
Revised: Therefore, in this paper, we devise a new revocable framework RGM to meet the characteristic of the graph matching problem, which is more friendly to graph with relatively more constraints.

Original: To verify the effectiveness of our proposed revocable framework, we compare it with the local rewrite framework in our experiments in Sec.
Revised: To verify the effectiveness of our proposed revocable framework, we compare it with the local rewrite framework in our experiments in Sec.

Original: A.4.6.
Revised: A.4.6.

Original: In fact, we believe our devised scheme for revocable action is suited to the setting when the graph size is moderate to afford such a costly scheme, while the constraints are relatively heavy and complex to make the revocable action necessary, whereby graph matching has become a suited problem setting.
Revised: We believe our devised scheme for revocable action is suited to the setting when the graph size is moderate to afford such a costly scheme, while the constraints are relatively heavy and complex to make the revocable action necessary, whereby graph matching has become a suited problem setting.

Original: A. E XPERIMENTS PROTOCOL
Revised: A. E XPERIMENTS PROTOCOL

Original: We perform experiments on various benchmarks including image data as well as pure combinatorial optimization problem instances, as the latter is especially suited for our back-end solver.
Revised: We perform experiments on various benchmarks including image data as well as pure combinatorial optimization problem instances, as the latter is especially suited for our back-end solver.

Original: We evaluate the robustness against outliers, as the (ground truth or estimated) number of inliers is given as hyperparameter.
Revised: We evaluate the robustness against outliers, as the (ground truth or estimated) number of inliers is given as hyperparameter.

Original: Whenever this information is known or not, one can always apply our proposed regularization technique to improve its robustness against outlier.
Revised: Whenever this information is known or not, one can always apply our proposed regularization technique to improve its robustness against outlier.

Original: The experiments are conducted on a Linux workstation with NVIDIA 2080Ti GPU and AMD Ryzen Threadripper 3970X 32-Core
Revised: The experiments are conducted on a Linux workstation with NVIDIA 2080Ti GPU and AMD Ryzen Threadripper 3970X 32-Core

Original: CPU with 128G RAM.
Revised: CPU with 128G RAM.

Original: Note that the graph size in our experiments is mostly more than 11 and the enumeration of all permutations is at the scale of more than 20 million which means an exhaustive search is impossible on commodity computers.
Revised: Note that the graph size in our experiments is mostly more than 11 and the enumeration of all permutations is at the scale of more than 20 million which means an exhaustive search is impossible on commodity computers.

Original: A.3. H YPERPARAMETER S ETTINGS
Revised: A.3. H YPERPARAMETER S ETTINGS

Original: For the hyper parameters, we set γ = 0 .
Revised: For the hyper parameters, we set γ = 0 .

Original: 9 in Eq.
Revised: 9 in Eq.

Original: 12, the target network update frequency as 40, the replay size as 100,000.
Revised: 12, the target network update frequency as 40, the replay size as 100,000.

Original: For the greedy part, the greedy rate (cid:15) decays from 1.0 to 0.02 in 20, episodes.
Revised: For the greedy part, the greedy rate (cid:15) decays from 1.0 to 0.02 in 20, episodes.

Original: For the learning module, we set 1e-5 as the learning rate and 64 as the batch size.
Revised: For the learning module, we set 1e-5 as the learning rate and 64 as the batch size.

Original: The hidden size of our GNN is 128 and the number of layers T is 3.
Revised: The hidden size of our GNN is 128 and the number of layers T is 3.

Original: The hidden size of the Q value network is 64.
Revised: The hidden size of the Q value network is 64.

Original: For the afﬁnity regularization module, the range of the data points used for approximation is
Revised: For the afﬁnity regularization module, the range of the data points used for approximation is

Original: S =
Revised: S =

Original: [ n x − 2 , n x + 2] .
Revised: [ n x − 2 , n x + 2] .

Original: A.3. E VALUATION M ETRICS
Revised: A.3. E VALUATION M ETRICS

Original: For testing, given the afﬁnity matrix K , RGM predicts a permutation matrix X pred ∈ { 0 , 1 } n 1 × ntransformed from its solution set U .
Revised: For testing, given the afﬁnity matrix K , RGM predicts a permutation matrix X pred ∈ { 0 , 1 } n 1 × ntransformed from its solution set U .

Original: Based on X pred and ground truth X gt ∈ { 0 , 1 } n 1 × n 2 (note that (cid:80)
Revised: Based on X pred and ground truth X gt ∈ { 0 , 1 } n 1 × n 2 (note that (cid:80)

Original: X gt equals the number of inliers, since the rows and columns of outliers are always zeros.).
Revised: X gt equals the number of inliers, since the rows and columns of outliers are always zeros.).

Original: Two evaluation metrics are used: objective afﬁnity score, and F1 score:
Revised: Two evaluation metrics are used: objective afﬁnity score, and F1 score:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original:  where ∗ denotes element-wise matrix multiplication.
Revised: (16) where ∗ denotes element-wise matrix multiplication.

Original: Note that the deﬁned objective score here is agnostic to the presence of outliers, which is a common protocol used in existing works.
Revised: Note that the deﬁned objective score here is agnostic to the presence of outliers, which is a common protocol used in existing works.

Original: Speciﬁcally, for a traditional afﬁnity matrix as used in previous works, its elements are set non-negative, and thus the solver generally aims to match node correspondences as many as possible.
Revised: Speciﬁcally, for a traditional afﬁnity matrix as used in previous works, its elements are set non-negative, and thus the solver generally aims to match node correspondences as many as possible.

Original: We also conduct experiments on the well-known QAPLIB dataset.
Revised: We also conduct experiments on the well-known QAPLIB dataset.

Original: For the problem instances on
Revised: For the problem instances on

Original: QAPLIB, the goal is to minimize the objective score.
Revised: QAPLIB, the goal is to minimize the objective score.

Original: Besides, the ground truth solution is supposed unknown due to its NP-hard nature.
Revised: Besides, the ground truth solution is supposed unknown due to its NP-hard nature.

Original: Therefore, we use the gap between the score of the predicted solution and the optimal score provided in the benchmark which is continuously updated by uploaded new best solutions, as the metric:
Revised: Therefore, we use the gap between the score of the predicted solution and the optimal score provided in the benchmark which is continuously updated by uploaded new best solutions, as the metric:

Original: [Equation]
Revised: [Equation]

Original: Note that in the QAP test, there is no outlier issue, and our solver purely optimizes the objective score, by matching all the nodes.
Revised: Note that in the QAP test, there is no outlier issue, and our solver purely optimizes the objective score, by matching all the nodes.

Original: A.3. C OMPARED M ETHODS
Revised: A.3. C OMPARED M ETHODS

Original: As mentioned before, RGM falls in line with learning-free graph matching back-end solvers that use the afﬁnity matrix K as input, regardless K is obtained by learning-based methods or not.
Revised: As mentioned before, RGM falls in line with learning-free graph matching back-end solvers that use the afﬁnity matrix K as input, regardless K is obtained by learning-based methods or not.

Original: Both traditional methods and learning-based methods are compared:
Revised: Both traditional methods and learning-based methods are compared:

Original: GAGM (Gold et al., 1996) utilizes the graduated assignment technique with an annealing scheme, which can iteratively approximate the cost function by Taylor expansion.
Revised: GAGM (Gold et al., 1996) utilizes the graduated assignment technique with an annealing scheme, which can iteratively approximate the cost function by Taylor expansion.

Original: RRWM (Cho et al., 2010) proposes a random walk view of the graph matching, with a re-weighted jump on graph matching.
Revised: RRWM (Cho et al., 2010) proposes a random walk view of the graph matching, with a re-weighted jump on graph matching.

Original: IFPF (Leordeanu et al., 2009) iteratively improves the solution via integer projection, given a continuous or discrete solution.
Revised: IFPF (Leordeanu et al., 2009) iteratively improves the solution via integer projection, given a continuous or discrete solution.

Original: PSM (Egozi et al., 2013) improves the spectral matching by presenting a probabilistic interpretation of the spectral relaxation scheme.
Revised: PSM (Egozi et al., 2013) improves the spectral matching by presenting a probabilistic interpretation of the spectral relaxation scheme.

Original: GNCCP (Liu et al., 2012) follows the convex-concave path-following scheme, with a simpler form of the partial permutation matrix.
Revised: GNCCP (Liu et al., 2012) follows the convex-concave path-following scheme, with a simpler form of the partial permutation matrix.

Original: BPF (Wang et al., 2018) designs a branch switching technique to seek better paths at the singular points, to deal with the singular point issue in the previous path following strategy.
Revised: BPF (Wang et al., 2018) designs a branch switching technique to seek better paths at the singular points, to deal with the singular point issue in the previous path following strategy.

Original: ZACR
Revised: ZACR

Original: (Wang et al., 2020a) designs to suppress the matching of outliers by assigning zero-valued vectors to the potential outliers, which is the latest graph matching solver designated for outliers.
Revised: (Wang et al., 2020a) designs to suppress the matching of outliers by assigning zero-valued vectors to the potential outliers, which is the latest graph matching solver designated for outliers.

Original: In particular, we further compare RGM with current popular deep graph matching methods:
Revised: In particular, we further compare RGM with current popular deep graph matching methods:

Original: GMN (Zanﬁr et al., 2018), PCA (Wang et al., 2019b), NGM (Wang et al., 2021a), LCS (Wang et al., 2020c), BBGM (Rol´ınek et al., 2020), which are the state-of-the-art deep graph matching methods, and more importantly, most of them are all open-sourced which are more convenient for a fair comparison.
Revised: GMN (Zanﬁr et al., 2018), PCA (Wang et al., 2019b), NGM (Wang et al., 2021a), LCS (Wang et al., 2020c), BBGM (Rol ´ ınek et al., 2020), which are the state-of-the-art deep graph matching methods, and more importantly, most of them are all open-sourced which are more convenient for a fair comparison.

Original: A.3. D ATASETS FOR E VALUATION
Revised: A.3. D ATASETS FOR E VALUATION

Original: We brieﬂy describe the used datasets, in line with the recent comprehensive evaluation for deep
Revised: We brieﬂy describe the used datasets, in line with the recent comprehensive evaluation for deep

Original: GM (Wang et al., 2021a).
Revised: GM (Wang et al., 2021a).

Original: Synthetic Dataset is created by random 2D coordinates as nodes and their distances as edge features for graph matching.
Revised: Synthetic Dataset is created by random 2D coordinates as nodes and their distances as edge features for graph matching.

Original: Speciﬁcally, one graph is randomly constructed to which random noise is further added to generate the other graph for matching.
Revised: Speciﬁcally, one graph is randomly constructed to which random noise is further added to generate the other graph for matching.

Original: The ground truth matching normally is set as the identity matrix.
Revised: The ground truth matching normally is set as the identity matrix.

Original: Willow Object is collected from real images by Cho et al.
Revised: Willow Object is collected from real images by Cho et al.

Original: It contains 256 images fromcategories, and each category is represented with at least 40 images.
Revised: It contains 256 images fromcategories, and each category is represented with at least 40 images.

Original: All instances in the same class[Figure] share 10 distinctive image keypoints whose correspondences are manually labeled as ground truth.
Revised: All instances in the same class share 10 distinctive image keypoints whose correspondences are manually labeled as ground truth.

Original: For testing the performance of handling outliers, we add several random outliers to each image.
Revised: For testing the performance of handling outliers, we add several random outliers to each image.

Original: Pascal VOC (Bourdev & Malik, 2009) consists of 20 classes with keypoint labels on natural images.
Revised: Pascal VOC (Bourdev & Malik, 2009) consists of 20 classes with keypoint labels on natural images.

Original: The instances vary by scale, pose and illumination.
Revised: The instances vary by scale, pose and illumination.

Original: The number of keypoints in each image ranges from 6 to 23.
Revised: The number of keypoints in each image ranges from 6 to 23.

Original: QAPLIB (Burkard et al., 1997) contains 134 real-world QAP instances from 15 categories, e.g. planning a hospital facility layout or testing of self-testable sequential circuits.
Revised: QAPLIB (Burkard et al., 1997) contains 134 real-world QAP instances from 15 categories, e.g. planning a hospital facility layout or testing of self-testable sequential circuits.

Original: The problem size is deﬁned as n 1 = n 2 by Lawler’s QAP.
Revised: The problem size is deﬁned as n 1 = n 2 by Lawler’s QAP.

Original: We use 14 of the 15 categories, the only one left is “els”, as there is only one sample in this category. 
Revised: We use 14 of the 15 categories, the only one left is “els”, as there is only one sample in this category. [Figure]

Original: A.3. T RAINING AND T ESTING P ROTOCOLS
Revised: A.3. T RAINING AND T ESTING P ROTOCOLS

Original: Due to the complexity in evaluation of learning-based graph matching methods and QAP solvers, here we elaborate the training and testing protocols in detail.
Revised: Due to the complexity in evaluation of learning-based graph matching methods and QAP solvers, here we elaborate the training and testing protocols in detail.

Original: We use the open-source version of the compared methods, and we tune or follow the hyperparameters set by the authors, to achieve the sound performance.
Revised: We use the open-source version of the compared methods, and we tune or follow the hyperparameters set by the authors, to achieve the sound performance.

Original: For the synthetic dataset, we use geometry features to construct the afﬁnity matrix with the train-test split rate (2 : 1) follows Jiang et al.(2021b).
Revised: For the synthetic dataset, we use geometry features to construct the afﬁnity matrix with the train-test split rate (2 : 1) follows Jiang et al.

Original: For the natural image dataset Willow Object and Pascal VOC, we use the pretrained features by
Revised: For the natural image dataset Willow Object and Pascal VOC, we use the pretrained features by

Original: CNN and GNN from BBGM (Rol ´ ınek et al., 2020) via supervised learning on the training set, and use the pre-splited train/test set (8 : 1) in line with BBGM as well.
Revised: CNN and GNN from BBGM (Rol ´ ınek et al., 2020) via supervised learning on the training set, and use the pre-splited train/test set (8 : 1) in line with BBGM as well.

Original: We input the learned afﬁnity matrix to RGM and all learning-free methods 2 , to make the comparison with supervised methods as fair as possible.
Revised: We input the learned afﬁnity matrix to RGM and all learning-free methods 2 , to make the comparison with supervised methods as fair as possible.

Original: For QAPLIB, there is no need for front-end feature extractors as the afﬁnity matrix is already given.
Revised: For QAPLIB, there is no need for front-end feature extractors as the afﬁnity matrix is already given.

Original: The train-test split rate for RGM is (1 : 1) for each of the selected 14 categories, as we choose the smaller-size half of the instances to train RGM in that category.
Revised: The train-test split rate for RGM is (1 : 1) for each of the selected 14 categories, as we choose the smaller-size half of the instances to train RGM in that category.

Original: While for the peer method
Revised: While for the peer method

Original: NGM (Wang et al., 2021a), due to its model’s nature, it does not split the train-test set in their
Revised: NGM (Wang et al., 2021a), due to its model’s nature, it does not split the train-test set in their

Original: QAPLIB experiments and test directly after training on the same set.
Revised: QAPLIB experiments and test directly after training on the same set.

Original: In contrast, our RGM follows the basic protocol in RL, which splits the train-test set to make a relatively fair comparison with the baselines.
Revised: In contrast, our RGM follows the basic protocol in RL, which splits the train-test set to make a relatively fair comparison with the baselines.

Original: A. A DDITIONAL E XPERIMENTS A.4. E XPERIMENTS ON S YNTHETIC D ATASET
Revised: A. A DDITIONAL E XPERIMENTS A.4. E XPERIMENTS ON S YNTHETIC D ATASET

Original: We evaluate RGM on the synthetic graphs following the protocol of Wang et al.
Revised: We evaluate RGM on the synthetic graphs following the protocol of Wang et al.

Original: (2021a).
Revised: (2021a).

Original: The synthetic data test is relatively simple, and it is mainly to show the effectiveness of our back-end solver when the front-end information is limited as there is no visual data for CNN to learn.
Revised: The synthetic data test is relatively simple, and it is mainly to show the effectiveness of our back-end solver when the front-end information is limited as there is no visual data for CNN to learn.

Original: More outlier tests will be given on the real data.
Revised: More outlier tests will be given on the real data.

Original: [Table]

Original: We ﬁrst generate sets of random points in the 2D plane.
Revised: We ﬁrst generate sets of random points in the 2D plane.

Original: The coordinates of these points are sample from uniform distribution U (0 , 1) × U (0 , 1) .
Revised: The coordinates of these points are sample from uniform distribution U (0 , 1) × U (0 , 1) .

Original: First, we select 10 sets of points as the set of ground truth points.
Revised: First, we select 10 sets of points as the set of ground truth points.

Original: Then, we randomly scale their coordinates from U (1 − δ s , 1 + δ s ) .
Revised: Then, we randomly scale their coordinates from U (1 − δ s , 1 + δ s ) .

Original: The set of scaled points and the set of ground truth points are regarded as the pairwise graphs to be matched.
Revised: The set of scaled points and the set of ground truth points are regarded as the pairwise graphs to be matched.

Original: We set there are 10 inliners without outlier, and the noise level δ s varies from 0 to 0 .
Revised: We set there are 10 inliners without outlier, and the noise level δ s varies from 0 to 0 .

Original: For the calculation of afﬁnity matrix K , the node afﬁnity is set by 0 and the edge afﬁnity is set by the difference of edge length: K ia,jb = exp( − ( f ij − f ab ) 2 σ) , where the f ij is the edge length of E ij .
Revised: For the calculation of afﬁnity matrix K , the node afﬁnity is set by 0 and the edge afﬁnity is set by the difference of edge length: K ia,jb = exp( − ( f ij − f ab ) 2 σ) , where the f ij is the edge length of E ij .

Original: We generate 300 sets of scaled points for each ground truth sets and get 3,000 pairwise graphs.
Revised: We generate 300 sets of scaled points for each ground truth sets and get 3,000 pairwise graphs.

Original: We split the data into the training and testing sets by the ratio of (2 : 1).
Revised: We split the data into the training and testing sets by the ratio of (2 : 1).

Original: The results of the synthetic datasets are shown in Fig.
Revised: The results of the synthetic datasets are shown in Fig.

Original: Evaluations are performed in terms of the noise level δ s .
Revised: Evaluations are performed in terms of the noise level δ s .

Original: We can see that RGM performs the best in terms of matching F1 score and objective score in all experiments. 
Revised: We can see that RGM performs the best in terms of matching F1 score and objective score in all experiments. [Table] [Table]

Original: A.4. P ERFORMANCE ON THE W ILLOW O BJECT D ATASET
Revised: P ERFORMANCE ON THE W ILLOW O BJECT D ATASET

Original: Table 6 shows the results over the ﬁve categories give three outliers.
Revised: Table 6 shows the results over the ﬁve categories give three outliers.

Original: We compare our methods (bottom box) with the learning-free back-end solvers (top box) and the learning based deep graph matching methods (middle box).
Revised: We compare our methods (bottom box) with the learning-free back-end solvers (top box) and the learning based deep graph matching methods (middle box).

Original: We use learning features extracted by BBGM (Rol´ınek et al., 2020) to construct the afﬁnity matrix, which is used as input for all learning-free back-end solvers and our methods.
Revised: We use learning features extracted by BBGM (Rol ´ ınek et al., 2020) to construct the afﬁnity matrix, which is used as input for all learning-free back-end solvers and our methods.

Original: For other learning-based baselines, we train and test them by their own pipelines directly, and they do not report their objective score because learning-based methods only care about the accuracy or F1 score.
Revised: For other learning-based baselines, we train and test them by their own pipelines directly, and they do not report their objective score because learning-based methods only care about the accuracy or F1 score.

Original: Since there are several outliers that should not be matched, the ground truth matching results only contain parts of the input keypoints (10 actually).
Revised: Since there are several outliers that should not be matched, the ground truth matching results only contain parts of the input keypoints (10 actually).

Original: Therefore, we use the F score instead of accuracy to test the performance.
Revised: Therefore, we use the F score instead of accuracy to test the performance.

Original: One may attribute the advantage of “RGM + IC” and “RGM + AR + IC” to the use extra information of the inlier count which is a unique ability of our RL-based model compared to peer baselines.
Revised: One may attribute the advantage of “RGM + IC” and “RGM + AR + IC” to the use extra information of the inlier count which is a unique ability of our RL-based model compared to peer baselines.

Original: Yet “RGM + AR” does not require the inlier count information as input, and can still outperform all baselines in almost all settings.
Revised: Yet “RGM + AR” does not require the inlier count information as input, and can still outperform all baselines in almost all settings.

Original: We ﬁnd a strange result that BPF (Wang et al., 2018) reaches the best performance rather than
Revised: We ﬁnd a strange result that BPF (Wang et al., 2018) reaches the best performance rather than

Original: ZACR
Revised: ZACR

Original: (Wang et al., 2020a), which is the latest GM solvers tailored for handling outliers.
Revised: (Wang et al., 2020a), which is the latest GM solvers tailored for handling outliers.

Revised: Per discussion with the authors of ZACR, this is perhaps mainly due to a few strong assumptions they made which is more suitable to the 50 images (30 cars and 20 motorbikes) they used for experiments in their paper, which may not always hold in other datasets including Willow Object.

Revised: For example, ZACR requires edges linked by two inliers have clear higher similarities than the edges linked by inlier-outlier or outlier-outlier.

Original: Per discussion with the authors of ZACR, this is perhaps mainly due to a few strong assumptions
Revised: Besides, by its inherent design, ZACR solves KoopmansBeckmann’s QAP instead of Lawler’s QAP, and therefore has some difﬁculty utilizing the afﬁnity matrix which is obtained by pretraining BBGM. Via the communication and discussion with the authors of ZACR, we have tried to modify their code including using the learned node and edge features inline with their model’s intereface. Regrettably, the results in Table 6 and Table 1 are the best results we can get. The matching visualization is given in Fig.

Original: they
Revised: We visualize the matching results of RGM on all ﬁve categories. We paint the inliers as green nodes and the outliers as blue nodes. In each pair of images, the green and red lines represent correct and incorrect predictions respectively. Since it is supposed to match all inliers and ignore the outliers, we can see that RGM barely matches the blue outliers and focuses on the green inliers.

Original: Car : 10/ Car : 10/
Revised: Car : 10/ Car : 10/

Original: Car : 6/
Revised: Car : 6/

Original: Car : 7/ Car : 8/
Revised: Car : 7/ Car : 8/

Original: Car : 10/
Revised: Car : 10/

Original: Duck : 8/
Revised: Duck : 8/

Original: Duck : 10/
Revised: Duck : 10/

Original: Duck : 9/ Duck : 7/
Revised: Duck : 9/ Duck : 7/

Original: Duck : 5/
Revised: Duck : 5/

Original: Duck : 9/ Face : 9/
Revised: Duck : 9/ Face : 9/

Original: Face : 10/
Revised: Face : 10/

Original: Face : 9/
Revised: Face : 9/

Original: Face : 10/
Revised: Face : 10/

Original: Face : 8/
Revised: Face : 8/

Original: Face : 10/
Revised: Face : 10/

Original: Motorbike : 8/ Motorbike : 9/
Revised: Motorbike : 8/ Motorbike : 9/

Original: Motorbike : 10/
Revised: Motorbike : 10/

Original: Motorbike : 8/ Motorbike : 7/
Revised: Motorbike : 8/ Motorbike : 7/

Original: Motorbike : 10/
Revised: Motorbike : 10/

Original: Winebottle : 10/
Revised: Winebottle : 10/

Original: Winebottle : 8/
Revised: Winebottle : 8/

Original: Winebottle : 9/
Revised: Winebottle : 9/

Original: Winebottle : 8/ Winebottle : 8/
Revised: Winebottle : 8/ Winebottle : 8/

Original: Winebottle : 10/
Revised: Winebottle : 10/

Revised: [Figure]

Revised: [Figure]

Revised: Testing

Original: [Figure]
Revised: [Figure]

Original: T r a n i n g
Revised: T r a n i n g

Original: A m o un t
Revised: A m o un t

Original: o f O u t li er s made which is more suitable to the 50 images (30 cars and 20 motorbikes) they used for experiments in their paper, which may not always hold in other datasets including Willow Object.

Original: For example, ZACR requires edges linked by two inliers have clear higher similarities than the edges linked by inlier-outlier or outlier-outlier.

Original: Besides, by its inherent design, ZACR solves KoopmansBeckmann’s QAP instead of Lawler’s QAP, and therefore has some difﬁculty utilizing the afﬁnity matrix which is obtained by pretraining BBGM.

Original: Via the communication and discussion with the authors of ZACR, we have tried to modify their code including using the learned node and edge features inline with their model’s intereface.

Original: Regrettably, the results in Table 6 and Table 1 are the best results we can get.

Original: The matching visualization is given in Fig. We visualize the matching results of RGM on all ﬁve categories. We paint the inliers as green nodes and the outliers as blue nodes. In each pair of images, the green and red lines represent correct and incorrect predictions respectively.
Revised: o f O u t li er s

Original: Since it is supposed to match all inliers and ignore the outliers, we can see that RGM barely matches the blue outliers and focuses on the green inliers.

Original: A.4. G ENERALIZATION
Revised: A.4. G ENERALIZATION

Original: 1) Generalization to different amounts of outliers.
Revised: 1) Generalization to different amounts of outliers.

Original: We carry additional experiments to test the generalization ability among different numbers of outliers.
Revised: We carry additional experiments to test the generalization ability among different numbers of outliers.

Original: In this study, we train our RGM on one certain number of outliers and test it on another setting.
Revised: In this study, we train our RGM on one certain number of outliers and test it on another setting.

Original: We conduct these experiments on Willow
Revised: We conduct these experiments on Willow

Original: Object with 10 inliers and a range of outliers from 0 to 6.
Revised: Object with 10 inliers and a range of outliers from 0 to 6.

Original: The results are shown in Fig.
Revised: The results are shown in Fig.

Original: 7, where for every testing case (column) darker red means better performance.
Revised: 7, where for every testing case (column) darker red means better performance.

Original: We can see that our RGM can generalize well to the different numbers of outliers, since the performance of RGM is promis- [Table] [Figure] [Table]
Revised: We can see that our RGM can generalize well to the different numbers of outliers, since the performance of RGM is promising.

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Figure]

Original: ing. Relatively speaking, training RGM with 2, 3, or 4 outliers can reach a better generalization performance [Figure] and training RGM with 3 outliers can reach the best.
Revised: Relatively speaking, training RGM with 2, 3, or 4 outliers can reach a better generalization performance, and training RGM with 3 outliers can reach the best.

Original: 2) Generalization among similar categories.
Revised: 2) Generalization among similar categories.

Original: Fig.
Revised: Fig.

Original: 8 (a) shows the generalization ability of RGM among similar categories.
Revised: 8 (a) shows the generalization ability of RGM among similar categories.

Original: We use one class for training and another class for testing.
Revised: We use one class for training and another class for testing.

Original: For every testing class (every column) the red is the darker the better.
Revised: For every testing class (every column) the red is the darker the better.

Original: We can see RGM generalizes well to different classes.
Revised: We can see RGM generalizes well to different classes.

Original: 2) Generalization on QAPLIB.
Revised: 2) Generalization on QAPLIB.

Original: Fig. 8(b) shows the generalization ability of RGM, which is trained on one class and tested on another.
Revised: Fig. 8(b) shows the generalization ability of RGM, which is trained on one class and tested on another.

Original: For every testing class (every column), the darker red means the low optimal gap, the better performance.
Revised: For every testing class (every column), the darker red means the low optimal gap, the better performance.

Original: It shows that RGM generalizes soundly to unseen instances with different problem sizes.
Revised: It shows that RGM generalizes soundly to unseen instances with different problem sizes.

Original: A.4. H YPERPARAMETER S ENSITIVITY S TUDY
Revised: A.4. H YPERPARAMETER S ENSITIVITY S TUDY

Original: We conduct the study on Willow Object with the same setting as aforementioned experiments, where there are 10 inliers and 3 outliers in each image.
Revised: We conduct the study on Willow Object with the same setting as aforementioned experiments, where there are 10 inliers and 3 outliers in each image.

Original: We choose six hyperparameters: batch size, γ , experience replay size, hidden size in GNN, hidden size in Q value network, and set the regularization function for afﬁnity regularization by three function forms.
Revised: We choose six hyperparameters: batch size, γ , experience replay size, hidden size in GNN, hidden size in Q value network, and set the regularization function for afﬁnity regularization by three function forms.

Original: The results in Fig.
Revised: The results in Fig.

Original: 9 show that RGM is not sensitive to hyperparameters in batch size, hidden size, and regularization function, since there are only small ﬂuctuations in F1 score.
Revised: 9 show that RGM is not sensitive to hyperparameters in batch size, hidden size, and regularization function, since there [Table] [Figure] [Figure]

Revised: [Figure]

Revised: different numbers of initial matched seeds (i.e. node pairs). are only small ﬂuctuations in F1 score.

Original: Here γ is the hyperparameter in Eq.
Revised: Here γ is the hyperparameter in Eq.

Original: 12 as the rate for considering current reward and long-term reward, whose value cannot be too high or too low.
Revised: 12 as the rate for considering current reward and long-term reward, whose value cannot be too high or too low.

Original: RGM performs badly when the replay size is too small for experience replay.
Revised: RGM performs badly when the replay size is too small for experience replay.

Original: A.4. C ASE S TUDY : S EEDED G RAPH M ATCHING
Revised: A.4. C ASE S TUDY : S EEDED G RAPH M ATCHING

Original: As aforementioned, the core of RGM is to learn the back-end decision making process for GM.
Revised: As aforementioned, the core of RGM is to learn the back-end decision making process for GM.

Original: Moreover, RGM can work ﬂexibly by utilizing additional information e.g. the initial seeds, which mean that one or several nodes in each of the original pairwise graphs is already matched by human[Figure] [Table] or other information sources.
Revised: Moreover, RGM can work ﬂexibly by utilizing additional information e.g. the initial seeds, which mean that one or several nodes in each of the original pairwise graphs is already matched by human or other information sources.

Original: In implementation, we only need to set the initial seeds as the ﬁrst several actions, and then let RGM execute normally.
Revised: In implementation, we only need to set the initial seeds as the ﬁrst several actions, and then let RGM execute normally.

Original: We conduct this case study on both the synthetic dataset and the Willow Object dataset, of which each image contains 10 inliers and 3 outliers.
Revised: We conduct this case study on both the synthetic dataset and the Willow Object dataset, of which each image contains 10 inliers and 3 outliers.

Original: Fig.
Revised: Fig.

Original: 10 shows that adding suitable initial seeds does improve the performance, which can be useful when the matching can be conducted by manual annotation in the beginning.
Revised: 10 shows that adding suitable initial seeds does [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Original: A.4. A BLATION S TUDY : R EVOCABLE A CTION V . S . L OCAL R EWRITE
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: improve the performance, which can be useful when the matching can be conducted by manual annotation in the beginning.

Revised: A.4. A BLATION S TUDY : R EVOCABLE A CTION V . S . L OCAL R EWRITE

Original: We study the effectiveness of our revocable action scheme, by comparing it with the local rewrite (LR (Chen & Tian, 2019)) under the same RL framework.
Revised: We study the effectiveness of our revocable action scheme, by comparing it with the local rewrite (LR (Chen & Tian, 2019)) under the same RL framework.

Original: LR is an inﬂuential mechanism in RLbased combinatorial optimization.
Revised: LR is an inﬂuential mechanism in RLbased combinatorial optimization.

Original: It tries to improve a given solution instead of generating one from scratch.
Revised: It tries to improve a given solution instead of generating one from scratch.

Original: To some extent, LR can also reverse the applied actions by its local rewrite mechanism and it is recognized as state-of-the-art technique for improving RL.
Revised: To some extent, LR can also reverse the applied actions by its local rewrite mechanism and it is recognized as state-of-the-art technique for improving RL.

Original: We conduct comparative experiments on QAPLIB.
Revised: We conduct comparative experiments on QAPLIB.

Original: We use LR to improve the solution given by the baselines and RGM without the revocable scheme (RGM w/o rev.), and compare the results with
Revised: We use LR to improve the solution given by the baselines and RGM without the revocable scheme (RGM w/o rev.), and compare the results with

Original: RGM.
Revised: RGM.

Original: Table 7 shows that our revocable framework RGM still performs the best compared to all boosted baselines.
Revised: Table 7 shows that our revocable framework RGM still performs the best compared to all boosted baselines.

Original: It turns out LR does improve the original solutions, but its performance and efﬁciency is still below our revocable framework.[Figure] [Table]
Revised: It turns out LR does improve the original solutions, but its performance and efﬁciency is still below our revocable framework.

Original: A.4. A BLATION S TUDY : U SING A LTERNATIVE RL B ACKBONES
Revised: A.4. A BLATION S TUDY : U SING A LTERNATIVE RL B ACKBONES

Original: In RGM [Table] we adopt Double Dueling DQN (D3QN) with priority experience replay as our backbone, for which we perform ablation study against alternatives on Willow Object with 10 inliers andoutliers.
Revised: In RGM, we adopt Double Dueling DQN (D3QN) with priority experience replay as our backbone, for which we perform ablation study against alternatives on Willow Object with 10 inliers andoutliers.

Original: Fig.
Revised: Fig.

Original: 11 shows the mean F1 score over ﬁve classes.
Revised: 11 shows the mean F1 score over ﬁve classes.

Original: We compare D3QN with popular backbones:
Revised: We compare D3QN with popular backbones:

Original: A2C (Mnih et al., 2016), ACER (Wang et al., 2017), TRPO (Schulman et al., 2015),
Revised: A2C (Mnih et al., 2016), ACER (Wang et al., 2017), TRPO (Schulman et al., 2015),

Original: PPO (Schulman et al., 2017), the original DQN (Mnih et al., 2013), and Double DQN (Hasselt et al., 2016).
Revised: PPO (Schulman et al., 2017), the original DQN (Mnih et al., 2013), and Double DQN (Hasselt et al., 2016).

Original: D3QN outperforms all other algorithms in the Willow Object.
Revised: D3QN outperforms all other algorithms in the Willow Object.

Original: We think that the main reason is that: graph matching is a discrete decision making problem, where the value based methods (DQN, DDQN, D3QN) can be more suitable than the policy based methods, which is widely accepted (Sutton & Barto, 2005; Ivanov & D’yakonov, 2019; Sutton & Barto, 2018).
Revised: We think that the main reason is that: graph matching is a discrete decision making problem, where the value based methods (DQN, DDQN, D3QN) can be more suitable than the policy based methods, which is widely accepted (Sutton & Barto, 2005; Ivanov & D’yakonov, 2019; Sutton & Barto, 2018).

Original: A.4. I NCONSISTENCY BETWEEN A FFINITY O BJECTIVE AND F 
Revised: A.4. I NCONSISTENCY BETWEEN A FFINITY O BJECTIVE AND F

Original: Finally, we discuss a standing issue in graph matching and possibly also in other optimization tasks regardless the presence of outliers.
Revised: Finally, we discuss a standing issue in graph matching and possibly also in other optimization tasks regardless the presence of outliers.

Original: The matching accuracy or F1 score is inconsistent with the value of objective function.
Revised: The matching accuracy or F1 score is inconsistent with the value of objective function.

Original: In our analysis, the reason is probably that the objective function cannot perfectly model the ultimate goal, due to limited modeling capacity and noise etc.
Revised: In our analysis, the reason is probably that the objective function cannot perfectly model the ultimate goal, due to limited modeling capacity and noise etc.

Original: For example, as shown in Table 8, when applying our RGM and RRWM (or any other method is ﬁne from the different-quality solution sampling perspective) on the real image dataset Pascal VOC, the resulting quantile statistics about the objective score deliver an important message: 39 .
Revised: For example, as shown in Table 8, when applying our RGM and RRWM (or any other method is ﬁne from the different-quality solution sampling perspective) on the real image dataset Pascal VOC, the resulting quantile statistics about the objective score deliver an important message: 39 .

Original: 4% sampled solutions by RGM can achieve an objective score even higher than one, which means these wrong solutions can even get a higher score than the ground truth matching.
Revised: 4% sampled solutions by RGM can achieve an objective score even higher than one, which means these wrong solutions can even get a higher score than the ground truth matching.

Original: Note that the front-end features
Revised: Note that the front-end features

Original: CNN/GNN and afﬁnity metric model are learned by the state-of-the-art supervised model BBGM, however the learned afﬁnity function seems still not perfectly ﬁt with the F1 score.
Revised: CNN/GNN and afﬁnity metric model are learned by the state-of-the-art supervised model BBGM, however the learned afﬁnity function seems still not perfectly ﬁt with the F1 score.

Original: In fact, this mismatch issue relates to the front-end afﬁnity learning and outlier dismissing, which is not the scope of the back-end solver as focused in this paper.
Revised: In fact, this mismatch issue relates to the front-end afﬁnity learning and outlier dismissing, which is not the scope of the back-end solver as focused in this paper.

Original: As Table 8 shows, though there are 39 .
Revised: As Table 8 shows, though there are 39 .

Original: 4% instances whose afﬁnity scores are larger than 1 , at the other 60 .
Revised: 4% instances whose afﬁnity scores are larger than 1 , at the other 60 .

Original: 6% instances, our RGM can solve threequarters of these instances perfectly.
Revised: 6% instances, our RGM can solve threequarters of these instances perfectly.

Original: Compared to RRWM, our RGM can solve more instances with the afﬁnity score equals to or nears to one, and that’s why our RGM can outperform RRWM and other baselines.
Revised: Compared to RRWM, our RGM can solve more instances with the afﬁnity score equals to or nears to one, and that’s why our RGM can outperform RRWM and other baselines.

Original: Despite its impressive results achieved in our paper, it also suggests the limitation of RL-based solvers which pursuits the high objective score which can sometimes be biased.
Revised: Despite its impressive results achieved in our paper, it also suggests the limitation of RL-based solvers which pursuits the high objective score which can sometimes be biased.

Original: One possibly mitigation way is involving multiple graphs (Yan et al., 2016; Jiang et al., 2021b) which we leave for future work.
Revised: One possibly mitigation way is involving multiple graphs (Yan et al., 2016; Jiang et al., 2021) which we leave for future work.