FEATURE-ROBUST OPTIMAL TRANSPORT
Year: 2020

Abstract:
Optimal transport is a machine learning problem with applications including distribution comparison, feature selection, and generative adversarial networks. In this paper, we propose feature-robust optimal transport (FROT) for highdimensional data, which solves high-dimensional OT problems using feature selection to avoid the curse of dimensionality. Specifically, we find a transport plan with discriminative features. To this end, we formulate the FROT problem as a min–max optimization problem. We then propose a convex formulation of the FROT problem and solve it using a Frank–Wolfe-based optimization algorithm, whereby the subproblem can be efficiently solved using the Sinkhorn algorithm. Since FROT finds the transport plan from selected features, it is robust to noise features. To show the effectiveness of FROT, we propose using the FROT algorithm for the layer selection problem in deep neural networks for semantic correspondence. By conducting synthetic and benchmark experiments, we demonstrate that the proposed method can find a strong correspondence by determining important layers. We show that the FROT algorithm achieves state-of-the-art performance in real-world semantic correspondence datasets.

heading: 1 INTRODUCTION
text: Optimal transport (OT) is a machine learning problem with several applications in the computer vision and natural language processing communities. The applications include Wasserstein distance estimation (Peyré et al., 2019), domain adaptation (Yan et al., 2018), multitask learning (Janati et al., 2019), barycenter estimation (Cuturi & Doucet, 2014), semantic correspondence (Liu et al., 2020), feature matching (Sarlin et al., 2019), and photo album summarization (Liu et al., 2019). The OT problem is extensively studied in the computer vision community as the earth mover’s distance (EMD) (Rubner et al., 2000). However, the computational cost of EMD is cubic and highly expensive. Recently, the entropic regularized EMD problem was proposed; this problem can be solved using the Sinkhorn algorithm with a quadratic cost (Cuturi, 2013). Owing to the development of the Sinkhorn algorithm, researchers have replaced the EMD computation with its regularized counterparts. However, the optimal transport problem for high-dimensional data has remained unsolved for many years.
Recently, a robust variant of the OT was proposed for high-dimensional OT problems and used for divergence estimation (Paty & Cuturi, 2019; 2020). In the robust OT framework, the transport plan is computed with the discriminative subspace of the two data matrices X ∈ Rd×n and Y ∈ Rd×m. The subspace can be obtained using dimensionality reduction. An advantage of the subspace robust approach is that it does not require prior information about the subspace. However, given prior information such as feature groups, we can consider a computationally efficient formulation. The computation of the subspace can be expensive if the dimensionality of data is high, for example, 104.
One of the most common prior information items is a feature group. The use of group features is popular in feature selection problems in the biomedical domain and has been extensively studied in Group Lasso (Yuan & Lin, 2006). The key idea of Group Lasso is to prespecify the group variables and select the set of group variables using the group norm (also known as the sum of `2 norms). For example, if we use a pretrained neural network as a feature extractor and compute OT using the features, then we require careful selection of important layers to compute OT. Specifically, each
layer output is regarded as a grouped input. Therefore, using a feature group as prior information is a natural setup and is important for considering OT for deep neural networks (DNNs).
In this paper, we propose a high-dimensional optimal transport method by utilizing prior information in the form of grouped features. Specifically, we propose a feature-robust optimal transport (FROT) problem, for which we select distinct group feature sets to estimate a transport plan instead of determining its distinct subsets, as proposed in (Paty & Cuturi, 2019; 2020). To this end, we formulate the FROT problem as a min–max optimization problem and transform it into a convex optimization problem, which can be accurately solved using the Frank–Wolfe algorithm (Frank & Wolfe, 1956; Jaggi, 2013). The FROT’s subproblem can be efficiently solved using the Sinkhorn algorithm (Cuturi, 2013). An advantage of FROT is that it can yield a transport plan from high-dimensional data using feature selection, using which the significance of the features is obtained without any additional cost. Therefore, the FROT formulation is highly suited for high-dimensional OT problems. Through synthetic experiments, we initially demonstrate that the proposed FROT is robust to noise dimensions (See Figure 1). Furthermore, we apply FROT to a semantic correspondence problem (Liu et al., 2020) and show that the proposed algorithm achieves SOTA performance.

heading: Contribution:
text: • We propose a feature robust optimal transport (FROT) problem and derive a simple and efficient Frank–Wolfe based algorithm. Furthermore, we propose a feature-robust Wasserstein distance (FRWD).
• We apply FROT to a high-dimensional feature selection problem and show that FROT is consistent with the Wasserstein distance-based feature selection algorithm with less computational cost than the original algorithm.
• We used FROT for the layer selection problem in a semantic correspondence problem and showed that the proposed algorithm outperforms existing baseline algorithms.

heading: 2 BACKGROUND
text: In this section, we briefly introduce the OT problem.
Optimal transport (OT): The following are given: independent and identically distributed (i.i.d.) samples X = {xi}ni=1 ∈ Rd×n from a d-dimensional distribution p, and i.i.d. samples Y = {yj}mj=1 ∈ Rd×m from the d-dimensional distribution q. In the Kantorovich relaxation of OT, admissible couplings are defined by the set of the transport plan:
U(µ, ν) = {Π ∈ Rn×m+ : Π1m = a,Π>1n = b},
where Π ∈ Rn×m+ is called the transport plan, 1n is the n-dimensional vector whose elements are ones, and a = (a1, a2, . . . , an)> ∈ Rn+ and b = (b1, b2, . . . , bm)> ∈ Rm+ are the weights. The OT problem between two discrete measures µ = ∑n i=1 aiδxi and ν = ∑m j=1 bjδyj determines the
optimal transport plan of the following problem:
min Π∈U(µ,ν)
n∑
i=1
m∑
j=1
πijc(xi,yj), (1)
where c(x,y) is a cost function. For example, the squared Euclidean distance is used, that is, c(x,y) = ‖x − y‖22. To solve the OT problem, Eq. (1) (also known as the earth mover’s distance) using linear programming requires O(n3), (n = m) computation, which is computationally expensive. To address this, an entropic-regularized optimal transport is used (Cuturi, 2013).
min Π∈U(µ,ν)
n∑
i=1
m∑
j=1
πijc(xi,yj) + H(Π),
where ≥ 0 is the regularization parameter, and H(Π) = ∑ni=1 ∑m j=1 πij(log(πij) − 1) is the entropic regularization. If = 0, then the regularized OT problem reduces to the EMD problem. Owing to entropic regularization, the entropic regularized OT problem can be accurately solved using Sinkhorn iteration (Cuturi, 2013) with a O(nm) computational cost (See Algorithm 1).
Wasserstein distance: If the cost function is defined as c(x,y) = d(x,y) with d(x,y) as a distance function and p ≥ 1, then we define the p-Wasserstein distance of two discrete measures µ = ∑n i=1 aiδxi and ν = ∑m j=1 bjδyj as
Wp(µ, ν) =
  min
Π∈U(µ,ν)
n∑
i=1
m∑
j=1
πijd(xi,yj) p
  1/p
.
Recently, a robust variant of the Wasserstein distance, called the subspace robust Wasserstein distance (SRW), was proposed (Paty & Cuturi, 2019). The SRW computes the OT problem in the discriminative subspace. This can be determined by solving dimensionality-reduction problems. Owing to the robustness, it can compute the Wasserstein from noisy data. The SRW is given as
SRW(µ, ν) =   min
Π∈U(µ,ν) max U∈Rd×k,U>U=Ik
n∑
i=1
m∑
j=1
πij‖U>xi −U>yj‖22
  1 2
, (2)
where U is the projection matrix with k ≤ d, and Ik ∈ Rk×k is the identity matrix. The SRW or its relaxed problem can be efficiently estimated using either eigenvalue decomposition or the Frank–Wolfe algorithm.

heading: 3 PROPOSED METHOD
text: This paper proposes FROT. We assume that the vectors are grouped as x = (x(1) > , . . . ,x(L) > )> and y = (y(1) > , . . . ,y(L) > )>. Here, x(`) ∈ Rd` and y(`) ∈ Rd` are the d` dimensional vectors, where ∑L `=1 d` = d. This setting is useful if we know the explicit group structure for the feature vectors a priori. In an application in L-layer neural networks, we consider x(`) and y(`) as outputs of the `th layer of the network. If we do not have a priori information, we can consider each feature independently (i.e., d1 = d2 = . . . = dL = 1 and L = d). All proofs in this section are provided in the Appendix.

heading: 3.1 FEATURE-ROBUST OPTIMAL TRANSPORT (FROT)
text: The FROT formulation is given by
min Π∈U(µ,ν) max α∈ΣL
n∑
i=1
m∑
j=1
πij
L∑
`=1
α`c(x (`) i ,y (`) j ), (3)
where ΣL = {α ∈ RL+ : α>1L = 1} is the probability simplex. The underlying concept of FROT is to estimate the transport plan Π using distinct groups with large distances between {x(`)i }ni=1 and {y(`)j }mj=1. We note that determining the transport plan in nondistinct groups is difficult because the data samples in {x(`)i }ni=1 and {y (`) j }mj=1 overlap. By contrast, in distinct groups, {x (`) i }ni=1 and {y(`)j }mj=1 are different, and this aids in determining an optimal transport plan. This is an intrinsically similar idea to the subspace robust Wasserstein distance (Paty & Cuturi, 2019), which estimates the transport plan in the discriminative subspace, while our approach selects important groups. Therefore, FROT can be regarded as a feature selection variant of the vanilla OT problem in Eq. (1), whereas the subspace robust version uses dimensionality-reduction counterparts.
Algorithm 1 Sinkhorn algorithm. 1: Input: a, b,C, , tmax 2: Initialize K = e−C/ ,u = 1n,v =
1m, t = 0 3: while t ≤ tmax and not converge do 4: u = a/(Kv) 5: v = b/(K>u) 6: t = t+ 1 7: end while 8: return Π = diag(u)Kdiag(v)
Algorithm 2 FROT with the Frank–Wolfe. 1: Input: {xi}ni=1, {yj}mj=1, η, and . 2: Initialize Π, compute {C`}L`=1. 3: for t = 0 . . . T do 4: Π̂ = argminΠ∈U(µ,ν)〈Π,MΠ(t)〉 +
H(Π)
5: Π(t+1) = (1− γ)Π(t) + γΠ̂ 6: with γ = 22+t . 7: end for 8: return Π(T )
Using FROT, we can define a p-feature robust Wasserstein distance (p-FRWD).
Proposition 1 For the distance function d(x,y),
FRWDp(µ, ν) =   min
Π∈U(µ,ν) max α∈ΣL
n∑
i=1
m∑
j=1
πij
L∑
`=1
α`d(x (`) i ,y (`) j ) p
  1/p
, (4)
is a distance for p ≥ 1.
Note that we can show that 2-FRWD is a special case of SRW with d(x,y) = ‖x − y‖2 (See Appendix). The key difference between SRW and FRWD is that FRWD can use any distance, while SRW can only use d(x,y) = ‖x− y‖2.

heading: 3.2 FROT OPTIMIZATION
text: Here, we propose two FROT algorithms based on the Frank–Wolfe algorithm and linear programming.
Frank–Wolfe: We propose a continuous variant of the FROT algorithm using the Frank–Wolfe algorithm, which can be fully differentiable. To this end, we introduce entropic regularization for α and rewrite the FROT as a function of Π. Therefore, we solve the following problem for α:
min Π∈U(µ,ν) max α∈ΣL
Jη(Π,α),with Jη(Π,α) = n∑
i=1
m∑
j=1
πij
L∑
`=1
α`c(x (`) i ,y (`) j )− ηH(α),
where η ≥ 0 is the regularization parameter, and H(α) = ∑L`=1 α`(log(α`) − 1) is the entropic regularization for α. An advantage of entropic regularization is that the nonnegative constraint is naturally satisfied, and the entropic regularizer is a strong convex function.
Lemma 2 The optimal solution of the optimization problem
α∗ = argmax α∈ΣL
Jη(Π,α), with Jη(Π,α) = L∑
`=1
α`φ` − ηH(α)
with a fixed admissible transport plan Π ∈ U(µ, ν), is given by
α∗` = exp
( 1 ηφ` )
∑L `′=1 exp ( 1 ηφ`′
) with Jη(Π,α∗) = η log ( L∑
`=1
exp
( 1
η φ`
)) + η.
Using Lemma 2 (or Lemma 4 in Nesterov (2005)) together with the setting φ` =∑n i=1 ∑m j=1 πijc(x (`) i ,y (`) i ) = 〈Π,C`〉, [C`]ij = c(x (`) i ,y (`) i ), the global problem is equivalent to
min Π∈U(µ,ν) Gη(Π), with Gη(Π) = η log
( L∑
`=1
exp
( 1
η 〈Π,C`〉
)) . (5)
Note that this is known as a smoothed max-operator (Nesterov, 2005; Blondel et al., 2018). Specifically, regularization parameter η controls the “smoothness” of the maximum.
Proposition 3 Gη(Π) is a convex function relative to Π.
The derived optimization problem of FROT is convex. Therefore, we can determine globally optimal solutions. Note that the SRW optimization problem is not jointly convex (Paty & Cuturi, 2019) for the projection matrix and the transport plan. In this study, we employ the Frank–Wolfe algorithm (Frank & Wolfe, 1956; Jaggi, 2013), using which we approximate Gη(Π) with linear functions at Π(t) and move Π toward the optimal solution in the convex set (See Algorithm 2).
The derivative of the loss function Gη(Π) at Π(t) is given by
∂Gη(Π)
∂Π ∣∣∣∣ Π=Π(t) = L∑
`=1
α (t) ` C` =MΠ(t) with α (t) ` =
exp (
1 η 〈Π(t),C`〉
)
∑L `′=1 exp ( 1 η 〈Π(t),C`′〉 ) .
Then, we update the transport plan by solving the EMD problem:
Π(t+1) = (1− γ)Π(t) + γΠ̂ with Π̂ = argmin Π∈U(µ,ν) 〈Π,MΠ(t)〉,
where γ = 2/(2+ k). Note thatMΠ(t) is given by the weighted sum of the cost matrices. Thus, we can utilize multiple features to estimate the transport plan Π for the relaxed problem in Eq. (5).
Using the Frank–Wolfe algorithm, we can obtain the optimal solution. However, solving the EMD problem requires a cubic computational cost that can be expensive if n and m are large. To address this, we can solve the regularized OT problem, which requires O(nm). We denote the Frank–Wolfe algorithm with EMD as FW-EMD and the Frank–Wolfe algorithm with Sinkhorn as FW-Sinkhorn.
Computational complexity: The proposed method depends on the Sinkhorn algorithm, which requires an O(nm) operation. The computation of the cost matrix in each subproblem needs an O(Lnm) operation, where L is the number of groups. Therefore, the entire complexity is O(TLnm), where T is the number of Frank–Wolfe iterations (in general, T = 10 is sufficient).
Proposition 4 For each t ≥ 1, the iteration Π(t) of Algorithm 2 satisfies
Gη(Π (t))−Gη(Π∗) ≤
4σmax(Φ >Φ)
η(t+ 2) (1 + δ),
where σmax(Φ>Φ) is the largest eigenvalue of the matrix Φ>Φ and Φ = (vec(C1), vec(C2), . . . , vec(CL))>; and δ ≥ 0 is the accuracy to which internal linear subproblems are solved.
Based on Proposition 4, the number of iterations depends on η, , and the number of groups. If we set a small η, convergence requires more time. In addition, if we use entropic regularization with a large , the δ in Proposition 4 can be large. Finally, if we use more groups, the largest eigenvalue of the matrix Φ>Φ can be larger. Note that the constant term of the upper bound is large; however, the Frank–Wolfe algorithm converges quickly in practice.
Linear Programming: Because limη→0+ Gη(Π) = max`∈{1,2,...,L} ∑n i=1 ∑m j=1 πijc(x (`) i ,y (`) j ), the FROT problem can also be written as
min Π∈U(µ,ν) max `∈{1,2,...,L}
n∑
i=1
m∑
j=1
πijc(x (`) i ,y (`) j ). (6)
Because the objective is the max of linear functions, it is convex with respect to Π. We can solve the problem via linear programming:
min Π∈U(µ,ν),t
t, s.t. 〈Π,C`〉 ≤ t, ` = 1, 2, . . . , L. (7)
This optimization can be easily solved using an off-the-shelf LP package. However, the computational cost of this LP problem is high in general (i.e., O(n3), n = m).

heading: 3.3 APPLICATION: SEMANTIC CORRESPONDENCE
text: We applied our proposed FROT algorithm to semantic correspondence. The semantic correspondence is a problem that determines the matching of objects in two images. That is, given input image pairs (A,B), with common objects, we formulated the semantic correspondence problem to estimate the transport plan from the key points in A to those in B; this framework was proposed in (Liu et al., 2020). In Figure 2, we show an overview of our proposed framework.
Cost matrix computation C`: In our framework, we employed a pretrained convolutional neural network to extract dense feature maps for each convolutional layer. The dense feature map of the `th layer output of the sth image is given by
f (`,s) s,q+(r−1)hs ∈ R d` , q = 1, 2, . . . , hs, r = 1, 2, . . . , ws, ` = 1, 2, . . . , L,
where ws and hs are the width and height of the sth image, respectively, and d` is the dimension of the `th layer’s feature map. Note that because the dimension of the dense feature map is different for each layer, we sample feature maps to the size of the 1st layer’s feature map size (i.e., hs ×ws). The `th layer’s cost matrix for images s and s′ is given by
[C`]ij = ‖f (`,s)i − f (`,s′) j ‖22, i = 1, 2, . . . , wshs, j = 1, 2, . . . , ws′hs′ .
Source image
Target image CAM
Feature Robust Optimal Transport (FROT)x (`) <latexit sha1_base64="L3BU/IT3q3uES8MG5EgG5Qq4+XQ=">AACLTXicvZ1LcxvHEYDHzsOO87AcX1KVC2NFiZRSVJCSlFNOpcp8ghIlEST4NiwWAC5BWMvFGgBJADT+gPIDcvApqfiQys/IJbeccvBPSOXopHJxpTKPnZ19zXQ3FjJZJMFB99c9Mz2PndkdtEK/OxhWKp+/8urXvv6Nb772+rfe+PZ3vvu9N2+89f29Qe+i3/Z22z2/1z9oNQee3w283WF36HsHYd9rnrd8b7/1fFm8v3/p9QfdXrAzHIfeh+fNTtA97babQ550fOPNRqvnn4yeXd9ueL5/Z3p842blXkV+LeRf3I9e3GTRV6331urbrMFOWI+12QU7Zx4L2JC/9lmTDfj3B+w+q7CQp33Irnlan7/qyvc9NmVvcN0LLuVxiSZPfc5/d/h/H0SpAf9fMAdSu82t+PynzzUX2K3KPyp/rnxR+VvlL5V/Vr60sq4lQ/gy5n9bStcLj9988YP6f0Gtc/53yM6MllPD56+Ef12pcy5zercgXdBceR+yU/ZrmecuL4NQpojSaCs/Lie//6L+3vat659U/lj5Fy+HP1Q+r/yVl0Rw+e/2Z1ve9qecLvgB17qSNj1p3+O2r3lteZzf5axrthK9DqRvXS4VRPVSrBvyVz3+e8r91BzhZy1K7/Ec4Ui+/HueIz2O0ptObf1/VnsnJecitOXrPGE5SvdlDPd51LgoHhvJSMlSVqP0MBHpxYRTWfvDHGEtTrfr6sjK6i4WRNwb7JajLEReuwX5WE68IwgLMUOUn6qngGtc83RNVTHS44ypbP8fyncDntKVsqqPEGktLrPAbkfR05P/qd/pelxgNznnDv87RfihY3QefhTHNc0fHenz8Me0jqQPDf49ix/B3D2BvdDtbh6lkW+rRaUCRYryoch6Q/a7I/5b+HCd8uGO7IuxfE+mKStnsnUJn3/M/1vi74/kq0seY2osEKPJg6h3hEt0i/c2KxG7z8cKX+q/K0fdaeLVNDUq5DkD3mcEEUePe33ZP+h33LnV/UYgezxFHEZjm2jHfgFZ6Uyl/G9BC6KOO3Jczfop4ilPT8o3+Eg8RVto8jqhWBDyOAt9+fp5KurTTCOh2uwpf++2jOuG7NE7XHIo489lZyjnSTYb6t3bUWuB6rUr5yZ2mpEo4/FA1lPbEoH6vQb7DcC5kO24H41m2uNPIn8+QWsPUvom/VqSpuwZb6NulvBZvKdq4kFE8qVER5aU6q/u8t8P4tjo5OYMReRu3J/ArazL7smfE/4zjbW6cf/izsGJnKMWt4ZQ5i7kr36U+NGpcJ8T8lcXkh2CfelJ1K+E7KeAbC2m1kBJTa2B1FD2JKpf+Jgdy97C56lnmblqXjOQ10ah7C2aUWRdR/Nj0bt/Nd+/4t/u2tC12+Lfy5kaF2nXMtWd16T8aiFjlcR4Wsh4SmJsFzK2SYx6IaOOiPGWnKGcsIlsFT3JSabrK9OebLsVxBiiNXvxWGvn3Y95GOIiQFskeLcEsJYIrGWAtUxgrQCsFQJrFWCtElhrAGuNwKoCrCqBtQ6w1gmshwDrIYH1CGA9IrA2ANYGgfUYYD0msJ4ArCcE1lOA9ZTA2gRYmwRWDWDVCKwtgLVFYG0DrG0Cqw6w6gTWDsDaIbB2AdYugbUHsPYIrH2AtU9gHQCsAwLrEGAdElhHAOuINHI3AVqT4FkLYLUIrDbAahNYJwDrhMCC5k0egXUKsE4JrA7A6hBYZwDrjMDqAqwugfURwPqIwHoOsJ4TWD7A8gmsc4B1TmAFACsgsKDrjx6BFQKskMD6GGB9TGD1AVafwBoArAGBNQRYQwLrAmBdEFiXAOuSwLoCWFcE1ghgjQisMcAaE1gTgDUhjdyTaDU2nfaMJVcbkqu2eC+H8Uo8xPUSHuNmG2bly1US6TUy/PzDk2uaENvI4UfXJtO7y252UhI/PxHrrxjPk5L4GUsoV23FPRDQiNDISeOjBlf2E3LZ47hUqtp1xpCTkvhZTw/FNnL4eUtT7lHD7KQkfibTlAz4uqCRkcXPb6ARqBHJ4Gc5MDEgEUfgrLURyeBnPDAxJBH7cjcFYmop/KylG+1LQuSkJL7NNRF1paXwMxpK/5aVxtfgGaoOz0i12EZR20SqyiHs64BEFemeHN8gclISS6+iR9ikJJa+gh5hk5L4lTXsWLIzw1jymNAnp2XxK11wtByQYqWGINZIxDq6Z6rP0DPtkvqQrDS+VDCtvUZs7TVUa68RW/smurUnJfFXjk151wGl1It1aBbxs768PM0SZsRPStLouNE/LUuzgJ8J5OWpJYVpFWlZZWHBaaUtryHNzrS+dlXpuD1pJbtkZWD2opXsspWB2YNWsitWBmbvWcmuWhmYPWclu2ZlYPaalWzVysDsMSvZdSsDs7esZB9aGZg9ZSX7yMrA7CUr2Q0rA7OHrGQfWxmYvWMl+8TKwOwZK9mnVgZmr1jJbloZmD1iJVuzMjB7w0p2y8rA7Akr2W0rA7MXrGTrVgZmD1jJ7lgZmL1fJbtrZWD2fJXsnpWB2etVsvtWBmaPV8keWBmYvV0le2hlYPZ0leyRlYHby9W5GSZ2MIpy5F4bxuXWZeNwLjYO4vVnWj48Yj7sNuz5SNpwWTHzRrOXeiXvvfXkmvYwIwX7bWSHOeJQzofFKyo1vxrq8rdo7ZRuAfK/jJUJIRf4lWg9k3RRzWwT6+sqykvKnYBVFJFyD98uiojp+V25xeTxI4c+Zd+/RYgQ+j5RixjjWQsuG8m+yNzRku6hThKkr+r+cPXt9l2tHjblcwH6CRkzBmM1xynNQ4LmJKV5BGiap2dPoqcdVMm6dMTKhFiLbsu/WuucrHUWeZqPyiILbnogY2sQ3esC+RIw85xCm1hCuk6z6ccFMYrZfc9yxgQ+Zkc+y5kQ+Lhd+r7UEft7yZUMYaUPxoTWnDpJOIYrnoq5WPI5/1H70S4/j6P+zvRzUHvwHSVQRHP3AeIJSvUM7jhXE9CzQ+psAvFkpLJ/ybwcA3pSKO3BFPDqGWvIFjskUfUOqItcFMl6PjubVX0vJNXqSWmrw0xMp8dWV53Pw5+iqDBlAcXMy/LAxMAsHlAjwVYG9t4GbkuukqFEiN0HV2RQLMwef9RWPVteXDVMbWvz8yBZl7fQPphRpmwvN6vdsv0cpcSzY6qrjF3UUD4/3YvvD1R7TG6Nnmz3XkZzVg883sK7kXYbUYLQSD6KR8/sbHIMlvHEojtB6OqZbfG+1AjUHzv1tX3oaWe96gY/7azXteCnncO4TEPLPP0uai4dxuVbzJmgOSOnNzhPXH5gCK6cUO71HbH83bkjgm72DtwRWJ/G4qjEXcDGto2C6dXGsS/jEr6MY19sFIwv6TsGTK2k02f3Msv3CHyM/8bnACHpRZJQtCjZi0jaNW7iSCGCRJtjwSNQsa0+aSabtTrbWE+dP5u9moMSkWd2SmwUytNoByzfY5XxLsnN9mbz8Hdc4G+Z3ibJzfpbpv+pykgSvCpK0j7j1iRormTGDDNnNmnlS36UmBNDXErJj4Arq+Q4ii2Di4IyuJhDGVwUlIGNSymDywJ/L+fg72WBvzYuxV9VvvaYNXLQniq+1MtYw+TNxEyZSDERAseFe3zVHNvoislTEOepzBgtRsTk+ZTmNCKXjjnBSGi0QBtGXmlAo6m4wtQnx2J2LwIGrSKKXEFl7Z5VCC117+QKOANRsrp+VkrEnGbpiLGxcLnQrcm+l5GUwbXg7Nq+2wZ2PpWnpFdSMLmhWsj3p64dkHKz1LyF/ChBsU6breYtwD3wPEv5zJrTsxI9YnGZ0m1Rxq90vrKrfa4onnc+y9um5Fs905BvMSq9fN4UJ98mXHya/wOL/4M5+T+w+G/nJ5+9tltYknuqnSitaBV0CXGi3Sk7ifZs+vK08eSabpq2yOrxaX/QCbSn1n7kFOHRhfTCnBkutLA9WlY73RpOidpQLtzrbTqqyq3aQTOaUakZsbmSLHP9OInzOo9zCuA8p68Y52ExkPeGlLOIW7NMzuL0/2XWKJMzTDeP0i+m1zjz/eO81ljt9nC1kX1qbF6rvckaehmryckam89qct5/+1zuZZaau97mZ9nMa4rm6/PgFs3EZ6+fxdjPxRLeLcZe2SgYXw5iX8qtUkP1DT+b4ObrvM5nFTwfJ/NZBcf3VuXKw1DykTmPEqoWlFB1Dv5WC/y1cSn+Lhb4W6ZtGUbe3zKtTTM2pL95nzcc1xl3S+ZF2cznh2qTnk+8PTwXnw/aflPe3/nsN+F7hnGpniEdWS8jntJx9DKiZwPlP0QxdVnu/gSo5srVl7mzt8z+o/2OSPjuJ/ddiPbcYfzSz38k751VKbOXmH4yJMs8LM08yjGPSjOTI7Q77/jyPMwxbXnHM49yTFveKUx3j2ee1IFWUGpMnztci7zSK0DwvSs1pvZoauiVozC2FpKt6VPbQrS1ILKGuZ+nyTD3COk4a2XiDuJruWZGD2sNU9sHiDI0fmCJ2LUdXSL4O+6MZDOni9Nc521H9yvF1+BGEj+jOENTz1Bxm+QmT9uDuNh5P9ZbypOrHTS1QyqDDroMMCduB0yvyfei1XWfqU9C06cz43ZodxjmeVbKCf8dFJFyqvg6ikiN8/nG4hKKiDn1x5VbTB5deaPkaJ1hn0WmlTyWSvF1BU2lfNbOPppK+3QILBVzlgjkI8YzqPQwZbYFMDBn3mw5IhejD0UsJk6h2sHUySPrdc8jhK5rdgLr71lt74G6a1bdNVC3atWtgrob0flT2auCjej0KUjXvg+BJbhnhBiKmb91Hf4kpbBEc545zE2ffU6j49aR8BYwfuNp5pRA1zhLOUsweQq1i1l0WjWmZOu564J64bUv5bNt6uwKxaR85sF+zs/90n7u5/wsZmL81P7N7pX2ZnYfTGn7yFr1ifXqI2vWJ9Zt3t/iUqD5u1/gr42Lq2PfWcs474xXZXy54un6E4SnUXxg5W093VXcb7ifJBZ3XGXvk06fiu3ypFjbnFbl0l1mq3yGsBmdjmc+h1yV3LKcPRzn0vfB0qwB5FoJtiZvknzeJPhcTLb7DLPXnKWxVqKc3eRaCbYgLzm9XrKWB9ZzF79W0sIin789lHVaRE++C3nqJtVyLBctfefgE8B2NepLFuO5sn7mS/R0t2X6HTAHmvIk/nThNEWk4ykXMqc2j1aJPrlYQYbmXv8QtA7T62PZvrgj4wha7Rb66nPO1ZOCHanzjP0MMeudxLbN/PaYVbg2ZHUgP2Ui+9nnKrrMe5Dnxn7ymQl3HYi0IPpcLmM5ebKyqomsV+KJGVMr+uy4BV5XvUh7gbdcUYpd/rslx6UBe0+ewK1ldUsQqeqUF9GqzPs7/P8D9nNpWZ80IWTfkSli1fOdhPSqzNV7xzdu3r9XkV8L9hd7D+7d/8W9B1u/vPn+ElNfr7Mfct5tdp+9y95n67xV7zJxBs2n7E/ssxd/f/GfF1+++J8SffWVSOdtlvr63Wv/Bx/eYmM=</latexit>
y(`) <latexit sha1_base64="YQVQlZDTAKyi4yW8oPOIIFbJkMw=">AACLTXicvZ1LcxvHEYDHzsOO87AcX1KVC2NFiZRSVJCSlFNOpcp8ghIlEST4NiwWAC5BWMvFGgBJADT+gPIDcvApqfiQys/IJbeccvBPSOXopHJxpTKPnZ19zXQ3FjJZJMFB99c9Mz2PndkdtEK/OxhWKp+/8urXvv6Nb772+rfe+PZ3vvu9N2+89f29Qe+i3/Z22z2/1z9oNQee3w283WF36HsHYd9rnrd8b7/1fFm8v3/p9QfdXrAzHIfeh+fNTtA97babQ550fOPNRqvnn4yfXd9ueL5/Z3p842blXkV+LeRf3I9e3GTRV6331urbrMFOWI+12QU7Zx4L2JC/9lmTDfj3B+w+q7CQp33Irnlan7/qyvc9NmVvcN0LLuVxiSZPfc5/d/h/H0SpAf9fMAdSu82t+PynzzUX2K3KPyp/rnxR+VvlL5V/Vr60sq4lQ/gy5n9bStcLj9988YP6f0Gtc/53yM6MllPD56+Ef12pcy5zercgXdBceR+yU/ZrmecuL4NQpojSaCs/Lie//6L+3vat659U/lj5Fy+HP1Q+r/yVl0Rw+e/2Z1ve9qecLvgB17qSNj1p3+O2r3lteZzf5axrthK9DqRvXS4VRPVSrBvyVz3+e8r91BzhZy1K7/Ec4Ui+/HueIz2O0ptObf1/VnsnJecitOXrPGE5SvdlDPd51LgoHhvJSMlSVqP0MBHpxYRTWfvDHGEtTrfr6sjK6i4WRNwb7JajLEReuwX5WE68IwgLMUOUn6qngGtc83RNVTHS44ypbP8fyncDntKVsqqPEGktLrPAbkfR05P/qd/pelxgNznnDv87RfihY3QefhTHNc0fHenz8Me0jqQPDf49ix/B3D2BvdDtbh6lkW+rRaUCRYryoch6Q/a7I/5b+HCd8uGO7IuxfE+mKStnsnUJn3/M/1vi74/kq0seY2osEKPJg6h3hEt0i/c2KxG7z8cKX+q/K0fdaeLVNDUq5DkD3mcEEUePe33ZP+h33LnV/UYgezxFHEZjm2jHfgFZ6Uyl/G9BC6KOO3Jczfop4ilPT8o3+Eg8RVto8jqhWBDyOAt9+fp5KurTTCOh2uwpf++2jOuG7NE7XHIo489lZyjnSTYb6t3bUWuB6rUr5yZ2mpEo4/FA1lPbEoH6vQb7DcC5kO24H41m2uNPIn8+QWsPUvom/VqSpuwZb6NulvBZvKdq4kFE8qVER5aU6q/u8t8P4tjo5OYMReRu3J/ArazL7smfE/4zjbW6cf/izsGJnKMWt4ZQ5i7kr36U+NGpcJ8T8lcXkh2CfelJ1K+E7KeAbC2m1kBJTa2B1FD2JKpf+Jgdy97C56lnmblqXjOQ10ah7C2aUWRdR/Nj0bt/Nd+/4t/u2tC12+Lfy5kaF2nXMtWd16T8aiFjlcR4Wsh4SmJsFzK2SYx6IaOOiPGWnKGcsIlsFT3JSabrK9OebLsVxBiiNXvxWGvn3Y95GOIiQFskeLcEsJYIrGWAtUxgrQCsFQJrFWCtElhrAGuNwKoCrCqBtQ6w1gmshwDrIYH1CGA9IrA2ANYGgfUYYD0msJ4ArCcE1lOA9ZTA2gRYmwRWDWDVCKwtgLVFYG0DrG0Cqw6w6gTWDsDaIbB2AdYugbUHsPYIrH2AtU9gHQCsAwLrEGAdElhHAOuINHI3AVqT4FkLYLUIrDbAahNYJwDrhMCC5k0egXUKsE4JrA7A6hBYZwDrjMDqAqwugfURwPqIwHoOsJ4TWD7A8gmsc4B1TmAFACsgsKDrjx6BFQKskMD6GGB9TGD1AVafwBoArAGBNQRYQwLrAmBdEFiXAOuSwLoCWFcE1ghgjQisMcAaE1gTgDUhjdyTaDU2nfaMJVcbkqu2eC+H8Uo8xPUSHuNmG2bly1US6TUy/PzDk2uaENvI4UfXJtO7y252UhI/PxHrrxjPk5L4GUsoV23FPRDQiNDISeOjBlf2E3LZ47hUqtp1xpCTkvhZTw/FNnL4eUtT7lHD7KQkfibTlAz4uqCRkcXPb6ARqBHJ4Gc5MDEgEUfgrLURyeBnPDAxJBH7cjcFYmop/KylG+1LQuSkJL7NNRF1paXwMxpK/5aVxtfgGaoOz0i12EZR20SqyiHs64BEFemeHN8gclISS6+iR9ikJJa+gh5hk5L4lTXsWLIzw1jymNAnp2XxK11wtByQYqWGINZIxDq6Z6rP0DPtkvqQrDS+VDCtvUZs7TVUa68RW/smurUnJfFXjk151wGl1It1aBbxs768PM0SZsRPStLouNE/LUuzgJ8J5OWpJYVpFWlZZWHBaaUtryHNzrS+dlXpuD1pJbtkZWD2opXsspWB2YNWsitWBmbvWcmuWhmYPWclu2ZlYPaalWzVysDsMSvZdSsDs7esZB9aGZg9ZSX7yMrA7CUr2Q0rA7OHrGQfWxmYvWMl+8TKwOwZK9mnVgZmr1jJbloZmD1iJVuzMjB7w0p2y8rA7Akr2W0rA7MXrGTrVgZmD1jJ7lgZmL1fJbtrZWD2fJXsnpWB2etVsvtWBmaPV8keWBmYvV0le2hlYPZ0leyRlYHby9W5GSZ2MIpy5F4bxuXWZeNwLjYO4vVnWj48Yj7sNuz5SNpwWTHzRrOXeiXvvfXkmvYwIwX7bWSHOeJQzofFKyo1vxrq8rdo7ZRuAfK/jJUJIRf4lWg9k3RRzWwT6+sqykvKnYBVFJFyD98uiojp+V25xeTxI4c+Zd+/RYgQ+j5RixjjWQsuG8m+yNzRku6hThKkr+r+cPXt9l2tHjblcwH6CRkzBmM1xynNQ4LmJKV5BGiap2dPoqcdVMm6dMTKhFiLbsu/WuucrHUWeZqPyiILbnogY2sQ3esC+RIw85xCm1hCuk6z6ccFMYrZfc9yxgQ+Zkc+y5kQ+Lhd+r7UEft7yZUMYaUPxoTWnDpJOIYrnoq5WPI5/1H70S4/j6P+zvRzUHvwHSVQRHP3AeIJSvUM7jhXE9CzQ+psAvFkpLJ/ybwcA3pSKO3BFPDqGWvIFjskUfUOqItcFMl6PjubVX0vJNXqSWmrw0xMp8dWV53Pw5+iqDBlAcXMy/LAxMAsHlAjwVYG9t4GbkuukqFEiN0HV2RQLMwef9RWPVteXDVMbWvz8yBZl7fQPphRpmwvN6vdsv0cpcSzY6qrjF3UUD4/3YvvD1R7TG6Nnmz3XkZzVg883sK7kXYbUYLQSD6KR8/sbHIMlvHEojtB6OqZbfG+1AjUHzv1tX3oaWe96gY/7azXteCnncO4TEPLPP0uai4dxuVbzJmgOSOnNzhPXH5gCK6cUO71HbH83bkjgm72DtwRWJ/G4qjEXcDGto2C6dXGsS/jEr6MY19sFIwv6TsGTK2k02f3Msv3CHyM/8bnACHpRZJQtCjZi0jaNW7iSCGCRJtjwSNQsa0+aSabtTrbWE+dP5u9moMSkWd2SmwUytNoByzfY5XxLsnN9mbz8Hdc4G+Z3ibJzfpbpv+pykgSvCpK0j7j1iRormTGDDNnNmnlS36UmBNDXErJj4Arq+Q4ii2Di4IyuJhDGVwUlIGNSymDywJ/L+fg72WBvzYuxV9VvvaYNXLQniq+1MtYw+TNxEyZSDERAseFe3zVHNvoislTEOepzBgtRsTk+ZTmNCKXjjnBSGi0QBtGXmlAo6m4wtQnx2J2LwIGrSKKXEFl7Z5VCC117+QKOANRsrp+VkrEnGbpiLGxcLnQrcm+l5GUwbXg7Nq+2wZ2PpWnpFdSMLmhWsj3p64dkHKz1LyF/ChBsU6breYtwD3wPEv5zJrTsxI9YnGZ0m1Rxq90vrKrfa4onnc+y9um5Fs905BvMSq9fN4UJ98mXHya/wOL/4M5+T+w+G/nJ5+9tltYknuqnSitaBV0CXGi3Sk7ifZs+vK08eSabpq2yOrxaX/QCbSn1n7kFOHRhfTCnBkutLA9WlY73RpOidpQLtzrbTqqyq3aQTOaUakZsbmSLHP9OInzOo9zCuA8p68Y52ExkPeGlLOIW7NMzuL0/2XWKJMzTDeP0i+m1zjz/eO81ljt9nC1kX1qbF6rvckaehmryckam89qct5/+1zuZZaau97mZ9nMa4rm6/PgFs3EZ6+fxdjPxRLeLcZe2SgYXw5iX8qtUkP1DT+b4ObrvM5nFTwfJ/NZBcf3VuXKw1DykTmPEqoWlFB1Dv5WC/y1cSn+Lhb4W6ZtGUbe3zKtTTM2pL95nzcc1xl3S+ZF2cznh2qTnk+8PTwXnw/aflPe3/nsN+F7hnGpniEdWS8jntJx9DKiZwPlP0QxdVnu/gSo5srVl7mzt8z+o/2OSPjuJ/ddiPbcYfzSz38k751VKbOXmH4yJMs8LM08yjGPSjOTI7Q77/jyPMwxbXnHM49yTFveKUx3j2ee1IFWUGpMnztci7zSK0DwvSs1pvZoauiVozC2FpKt6VPbQrS1ILKGuZ+nyTD3COk4a2XiDuJruWZGD2sNU9sHiDI0fmCJ2LUdXSL4O+6MZDOni9Nc521H9yvF1+BGEj+jOENTz1Bxm+QmT9uDuNh5P9ZbypOrHTS1QyqDDroMMCduB0yvyfei1XWfqU9C06cz43ZodxjmeVbKCf8dFJFyqvg6ikiN8/nG4hKKiDn1x5VbTB5deaPkaJ1hn0WmlTyWSvF1BU2lfNbOPppK+3QILBVzlgjkI8YzqPQwZbYFMDBn3mw5IhejD0UsJk6h2sHUySPrdc8jhK5rdgLr71lt74G6a1bdNVC3atWtgrob0flT2auCjej0KUjXvg+BJbhnhBiKmb91Hf4kpbBEc545zE2ffU6j49aR8BYwfuNp5pRA1zhLOUsweQq1i1l0WjWmZOu564J64bUv5bNt6uwKxaR85sF+zs/90n7u5/wsZmL81P7N7pX2ZnYfTGn7yFr1ifXqI2vWJ9Zt3t/iUqD5u1/gr42Lq2PfWcs474xXZXy54un6E4SnUXxg5W093VXcb7ifJBZ3XGXvk06fiu3ypFjbnFbl0l1mq3yGsBmdjmc+h1yV3LKcPRzn0vfB0qwB5FoJtiZvknzeJPhcTLb7DLPXnKWxVqKc3eRaCbYgLzm9XrKWB9ZzF79W0sIin789lHVaRE++C3nqJtVyLBctfefgE8B2NepLFuO5sn7mS/R0t2X6HTAHmvIk/nThNEWk4ykXMqc2j1aJPrlYQYbmXv8QtA7T62PZvrgj4wha7Rb66nPO1ZOCHanzjP0MMeudxLbN/PaYVbg2ZHUgP2Ui+9nnKrrMe5Dnxn7ymQl3HYi0IPpcLmM5ebKyqomsV+KJGVMr+uy4BV5XvUh7gbdcUYpd/rslx6UBe0+ewK1ldUsQqeqUF9GqzPs7/P8D9nNpWZ80IWTfkSli1fOdhPSqzNV7xzdu3r9XkV8L9hd7D+7d/8W9B1u/vPn+ElNfr7Mfct5tdp+9y95n67xV7zJxBs2n7E/ssxd/f/GfF1+++J8SffWVSOdtlvr63Wv/B6rAYmQ=</latexit>
C`<latexit sha1_base64="OacnRizJRBX89wDWgRojPha2Xaw=">AACLWnicvZ1Lb2NJFYBrhtc8gOlhWCDBIkzT0IOalrsBDRpAmjyd7nR3nDjv8XTkx43jaef6jh+J7Yw3LOkfwIIVSEggVvAX2PAHWIzEH0AsB4kNC+px69Z9VZ1zfN2TKIlTPuc7p6pOPW7VveVG0O0MhqXSpy+9/IUvfunLX3nl1dde/+rXvv7GjTe/cTDojfpNb7/Z6/b6R436wOt2fG9/2Bl2vaOg79UvGl3vsPFsVbx/eOn1B52evzecBN6HF/W23znrNOtDnnR64zu1oTceSs5132vNrmuNXre1elrzut3Z6Y2bpbsl+bWUfXEvfHGThV+V3pvrb7Eaa7Eea7IRu2Ae89mQv+6yOhvw7w/YPVZiAU/7kF3ztD5/1ZHve2zGXuO6Iy7lcYk6T33Gf7f5fx+EqT7/XzAHUrvJrXT5T59rLrFbpX+U/lT6rPT30p9L/yr9z8q6lgzhy4T/bShdLzh949ffqv4X1Lrgf4fs3Gg5Nbr8lfCvI3UuZE7v5KQLmivvQ3bGfibz3OFlEMgUURpN5cfl9DefVd/bvXX9/dLvS//m5fC70qelv/GS8C//0/zDjrf7W04XfJ9rXUmbnrTvcdvXvLY8zu9w1jVbC1/70rcOl/LDesnXDfirHv89435qjvCzEqb3eI5wpK78e5EhPQrT605t/X9aey8h5yI05essYTVM78oY7vOocVE8NpaRkqash+lBLNLzCWey9ocZwkaUbtfVkZXWXc6JuNfYLUdZiLx2cvKxGntHEJYihig/VU8+17jm6ZqqYqTHGTPZ/j+U7/o8pSNlVR8h0hpcZondDqOnJ/9Tv5P1uMRucs47/O8M4YeO0UX4kR/XNH90pC/CH9M64j7U+Pc8fvgL9wT2Qre7RZRGtq3mlQoUKcqHPOs12e+O+W/hw3XCh3dkX4zlezJNWTmXrUv4/D3+3wp/fyxfXfIYU2OBGE3uh70jXKI7vLdZC9l9PlZ0pf67ctSdxV7NEqNCljPgfYYfcvS415f9g37HnVvdb/iyx1PEYTi2iXbczSErnZmU/yVoQdRxW46raT9FPGXpcfkaH4lnaAt1XicUC0IeZ6EvXz9LRH2SaSRUmz3j792WcV2TPXqbSw5l/LnsDOU8yWZDvXs7bC1QvXbk3MROMxJFPB7IempaIlC/V2M/Bzgj2Y774WimPf4k9OcTtPYgoW/SryVpxp7yNupmCZ/Fe6om7oekrpRoy5JS/dUd/vt+FBvtzJwhj9yJ+hO4lXXYXfnT4j+zSKsT9S/uHLTkHDW/NQQydwF/9d3Yj06F+5yAvxpJdgD2pa2wXwnYDwDZSkStgJKaWgGpgexJVL/wMTuVvUWXp56n5qpZTV9eGwWyt6iHkXUdzo9F7/75fP+Uf7trQ9dug3+vpmpcpF3LVHde4/LruYx1EuNJLuMJibGby9glMaq5jCoixhtyhtJiU9kqepITT9dXpj3ZdkuIMURr9qKx1s67F/EwxGWAtkzwbgVgrRBYqwBrlcBaA1hrBNY6wFonsDYA1gaBVQZYZQJrE2BtElgPANYDAushwHpIYG0BrC0C6xHAekRgPQZYjwmsJwDrCYG1DbC2CawKwKoQWDsAa4fA2gVYuwRWFWBVCaw9gLVHYO0DrH0C6wBgHRBYhwDrkMA6AlhHBNYxwDomsE4A1glp5K4DtDrBswbAahBYTYDVJLBaAKtFYEHzJo/AOgNYZwRWG2C1CaxzgHVOYHUAVofA+ghgfURgPQNYzwisLsDqElgXAOuCwPIBlk9gQdcfPQIrAFgBgfUxwPqYwOoDrD6BNQBYAwJrCLCGBNYIYI0IrEuAdUlgXQGsKwJrDLDGBNYEYE0IrCnAmpJG7mm4GptMe8riqw3xVVu8l8NoJR7iejGPcbMNs/LlKonkGhl+/uHJNU2IbeTwo2ud6d1lNzsuiZ+fiPVXjOdxSfyMJZCrtuIeCGhEqGWk8VGDK/spuexxXCpV7TpjyHFJ/Kynh2IbOfy8pS73qGF2XBI/k6lLBnxdUEvJ4uc30AhUC2XwsxyY6JOIY3DWWgtl8DMemBiQiH25mwIxtRR+1tIJ9yUhclwS3+bqiLrSUvgZDaV/S0vja/AcVYfnpFpsoqhNIlXlEPZ1QKKKdE+ObxA5Lomll9EjbFwSS19Dj7BxSfzKGnYs2ZtjLHlE6JOTsviVLjhajkixUkEQKyRiFd0zVefomfZJfUhaGl8qmNZeIbb2Cqq1V4itfRvd2uOS+CvHurzrgFLq+To0i/hZX1aeZgkz4sclaXTc6J+UpVnAzwSy8tSSwrSKpKyysOS00pTXkGZnWl+7qnTcnrSSXbEyMHvRSnbVysDsQSvZNSsDs/esZNetDMyes5LdsDIwe81KtmxlYPaYleymlYHZW1ayD6wMzJ6ykn1oZWD2kpXslpWB2UNWso+sDMzesZJ9bGVg9oyV7BMrA7NXrGS3rQzMHrGSrVgZmL1hJbtjZWD2hJXsrpWB2QtWslUrA7MHrGT3rAzM3q+S3bcyMHu+SvbAysDs9SrZQysDs8erZI+sDMzerpI9tjIwe7pK9sTKwO3l6twMYzsYeTlyrw3jcuuycbwQG0fR+jMtHx4xH3Yb9nzEbbismHmj2Uu9kvfeenJNe5iSgv02ssMMcSjnw+IVlZpdDXX5m7d2SrcA+V/EypSQC/xKtJ5Juqhmton1dR3lJeVOwDKKSLmHbx9FxPT8rtxi8viRQ5+y798gRAh9n6hBjPG0BZeNeF9k7mhJ9lCtGOnzuj9cfbt9V6uHdflcgH5CxozBWM1JQvOYoDlNaJ4Amubp2Vb4tIMqWZeOWJkQa9FN+VdrXZC1zkNPs1GZZ8FN92VsDcJ7XSBffGaeU2gSS0jXaTr9NCdGMbvvac6EwMfsyKc5UwIft0vflzpify++kiGs9MGY0JozJwnHcMVTPhdLvuA/aj/a5edp2N+Zfg5qD11HCeTR3H2AeIJSPYM7ydQE9OyQOptAPBmp7F8yL8OAnhRKejADvHrKarLFDklUvQPqIudFsp7PzmdV3wtJtdoqbHWYiunk2Oqq80X4kxcVpiygmHlRHpgYmMcDaiTYysDe28BtyVUylAix++CKDIqF+eOP2qrny4urhqltbXEexOvyFtoHM8oU7eXmtVu0n6OUeHpMdZWxixrI56d70f2Bao/JrdGT7d5Lac7rgcdbeCfUbiJKEBrJx9HomZ5NTsAynlp0pwhdPbPN35cag/oTp762Dz3trFfd4Ked9boW/LRzEJVpYJmn30HNpYOofPM5UzRn7PQG54nLDwzBlRPKvb5jlr07d0zQTd+BOwbr01gcF7gL2Ni2UTC92iTyZVLAl0nki42C8SV5x4CplWT6/F6m+R6Bj/Hf+OwjJL1QEooWJTsKpV3jJo4UIEi0ORY8AuXb6pNmsmmr84311Pmz2as5KhB5ZqfERqE8jXbEsj1WEe/i3HRvtgh/Jzn+Fult4ty0v0X6n7KMJMEroyTtM25NguZKZswwc2aTVrzkx7E5McSllPwYuLKKj6PYMhjllMFoAWUwyikDG5dSBpc5/l4uwN/LHH9tXIq/qnztMWvkoD1VfKkXsYbJm4mZIpFiIgSOC/f4qjm20RWTJz/KU5ExWoyI8fMpzWlELh1zgpHQaIA2jLzSgEZTcYWpT47F7F74DFpFFLmCyto9qxBa6t7JNXAGomR1/awViDnN0hFjY+FyoVuTfS8jLoNrwem1fbcN7HwqS0mupGByQ7WQ7U9dOyDFZqlZC9lRgmKdNlvNWoB74EWW8rk1p+cFesT8MqXbooxfyXylV/tcUbzofBa3Tcm3eqYh22JUevG8KU62Tbj4NP8HFv8HC/J/YPHfzo8/e223sCL3VNthWt4q6AriRLsz1gr3bPrytPH4mm6Stsyq0Wl/0Am0Z9Z+5Azh0Uh6Yc4MF1rYHi2tnWwNZ0RtKBfu9TYdVcVW7aAZzbjQjNhcSRa5fpxGeV3EOQVwnpNXjIuw6Mt7Q4pZxK1Zxmdx+v8ia5TxGaabR+kXk2uc2f5xUWusdnu42kg/Nbao1d54Db2I1eR4jS1mNTnrv30u9yJLzV1vi7Ns5jV58/VFcPNm4vPXz3Lk53IB75Yjr2wUjC9HkS/FVqmh+oafTXDzdV4XswqejZPFrILje6ti5WEo2chcRAmVc0qovAB/yzn+2rgUf5dz/C3Stgwj62+R1qYZW9LfrM9bjuuMOwXzomxm80O1Sc8n3h6ei88Hbb8p6+9i9pvwPcOkUM+QjKwXEU/JOHoR0bOF8h+imLosdn8CVHPF6svc2Vtk/9F+RyR895P7LkR77jB+6ec/4vfOqpT5S0w/GZJmHhdmnmSYJ4WZ8RHanXd8eR5nmLa845knGaYt7xSmu8czT+pAKygVps8droRe6RUg+N6VClN7NBX0ylEQWQvI1vSpbQHamh9aw9zPU2eYe4R0nDVScQfxtVw9pYe1hqntI0QZGj+wROzaji4R/B13RrKe0cVpbvK2o/uV/GtwI4mfUZyjqeeouI1z46ftQVzsvB/rLeXJ1Taa2iaVQRtdBpgTt32m1+R74ep6l6lPQtOnM+N2aPcY5nlWygn/bRSRcqr4JopIjfPFxuIKiog59ceVW0weXXmj5GiTYZ9FppU8lkrxdQ1NpXzWziGaSvt0CCwVc5YI5CPGM6j0MGW2AzAwZ97sOCIXow9FLCZOodrB1MlD63XPQ4Sua3YC6x9YbR+AuhtW3Q1Qt2zVLYO6W+H5U+mrgq3w9ClI174PgSW4Z4QYipm/dRz+xKWwRHOeOcxNnn1Oo+PWkfAWMH7jaeaUQNc4SzlLMH4KtYuZd1o1pmSrmeuCau61L+WzbarsCsWkfObBYcbPw8J+Hmb8zGdi/NT+ze+V9mZ+H0xpd5G12iXWaxdZs11i3Wb9zS8Fmr+HOf7auLg67jprGeed8aqIL1c8XX+C8CyMD6y8rae7ivoN95PE4o6r9H3SyVOxXZ7ka5vTqly6q2ydzxC2w9PxzOeQq5JblbOH00z6IViaFYBcKcDW5G2Sz9sEn/PJdp9h9oazNDYKlLObXCnAFuQVp9cr1vLAeu7iVwpaWObztweyTvPo8XchT92kSobloiXvHHwM2C6HfclyNFfWz3yJnu62TH8HzIGmPI4+XThJEel4ykjm1ObROtEnF8tP0dzrH4LWZnp9LN0Xt2UcQavdQl99zrl6UrAtdZ6yHyJmvdPItpnfnrIS14asDuSnTKQ/+1xFl3kP8tzYjz8z4a4DkeaHn8tlLMdPVlY1kfZKPDFjakWfHbfE66oXai/xlitKscN/N+S4NGDvyRO4taxuCSJVnfIiWpV5f4//f8R+JC3rkyaE7NsyRax6vh2TXpe5eu/0xs17d0vya8n+4uD+3Xs/vnt/5yc33/8FU1+vsG9z3m12j73L3mebvFXv85z8iv2R/YX99df/fP7y81efv65EX34p1HmLJb6ef/P/i9dl0A==</latexit>
a <latexit sha1_base64="MoFVAJMAn0YI/DAGDrQZ+yOSN10=">AACLQnicvZ1JcyNJFYBzhm1olplhLkRwMdM09BBNh7qBGKIJIsar3O3utmx5H087JLksq1su1WixJXn0B/rEjQBOEMGB4GfAgT/AYX4CcCKGCC5zIJfKytoy33uq6rHDtpx673svM18ulVmVagbdzmBYqXz62utf+vJXvvq1N75+4xvf/Na333zr7e/sDXqjfsvbbfW6vf5BszHwuh3f2x12hl3vIOh7jYtm19tvvlgW7+9fev1Bp+fvDCeB99FFo+13zjqtxpAn7R03e93TxslbNyt3K/JrIfviXvjiJgu/ar23V99hx+yU9ViLjdgF85jPhvx1lzXYgH9/yO6xCgt42kfsmqf1+auOfN9jM3aD6464lMclGjz1Bf/d5v99GKb6/H/BHEjtFrfS5T99rrnAblX+Uflz5bPK3yt/qfyz8rmVdS0ZwpcJ/9tUul5w8ubL79b/B2pd8L9Ddm60nBpd/kr415E6FzKnd3LSBc2V9yE7Y7+Qee7wMghkiiiNlvLjcvqbz+oPtm9d/7Dyx8q/eTn8ofJp5a+8JPzL/7b+tOVt/57TBd/nWlfSpifte9z2Na8tj/M7nHXNVsLXvvStw6X8sF7ydQP+qsd/z7ifmiP8rIXpPZ4jHKkr/15kSI/D9IZTW/+f1t5JyLkILfk6S1gO07syhvs8alwUj41lpKQpq2F6EIv0fMKZrP1hhrAWpdt1dWSldRdzIu4Gu+UoC5HXTk4+lmPvCMJCxBDlp+rJ5xrXPF1TVYz0OGMm2/9H8l2fp3SkrOojRFqTyyyw22H09OR/6neyHhfYTc55j/+dIfzQMVqGH/lxTfNHR3oZ/pjWEffhmH/P44dfuiewF7rdlVEa2baaVypQpCgf8qwfy353zH8LH64TPrwn+2Is35Npysq5bF3C5x/w/5b4+2P56pLHmBoLxGhyP+wd4RLd4r3NSsju87GiK/Xfl6PuLPZqlhgVspwB7zP8kKPHvb7sH/Q77tzqfsOXPZ4iDsOxTbTjbg5Z6cyk/K9AC6KO23JcTfsp4ilLj8sf85F4hrbQ4HVCsSDkcRb68vWLRNQnmUZCtdkz/t5tGdfHskdvc8mhjD+XnaGcJ9lsqHdvh60FqteOnJvYaUaiiMcDWU8tSwTq947ZLwHOSLbjfjiaaY8/Cf35BK09SOib9GtJmrFnvI26WcJn8Z6qifshqSsl2rKkVH91h/++H8VGOzNnyCN3ov4EbmUddlf+nPKfWaTVifoXdw5O5Rw1vzUEMncBf/X92I9OhfucgL8aSXYA9qWnYb8SsB8BsrWIWgMlNbUGUgPZk6h+4WN2InuLLk89T81Vs5q+vDYKZG/RCCPrOpwfi979i/n+Of9214au3Sb/Xk7VuEi7lqnuvMblV3MZqyTG01zGUxJjO5exTWLUcxl1RIw35QzllE1lq+hJTjxdX5n2ZNutIMYQrdmLxlo7717EwxAXAdoiwbslgLVEYC0DrGUCawVgrRBYqwBrlcBaA1hrBFYVYFUJrHWAtU5gPQRYDwmsRwDrEYG1AbA2CKzHAOsxgfUEYD0hsJ4CrKcE1ibA2iSwagCrRmBtAawtAmsbYG0TWHWAVSewdgDWDoG1C7B2Caw9gLVHYO0DrH0C6wBgHRBYhwDrkMA6AlhHpJG7AdAaBM+aAKtJYLUAVovAOgVYpwQWNG/yCKwzgHVGYLUBVpvAOgdY5wRWB2B1CKznAOs5gfUCYL0gsLoAq0tgXQCsCwLLB1g+gQVdf/QIrABgBQTWxwDrYwKrD7D6BNYAYA0IrCHAGhJYI4A1IrAuAdYlgXUFsK4IrDHAGhNYE4A1IbCmAGtKGrmn4WpsMu0Zi682xFdt8V4Oo5V4iOvFPMbNNszKl6skkmtk+PmHJ9c0IbaRw4+uDaZ3l93suCR+fiLWXzGexyXxM5ZArtqKeyCgEeE4I42PGlzZT8llj+NSqWrXGUOOS+JnPT0U28jh5y0NuUcNs+OS+JlMQzLg64LjlCx+fgONQMehDH6WAxN9EnEMzlqPQxn8jAcmBiRiX+6mQEwthZ+1dMJ9SYgcl8S3uQairrQUfkZD6d/S0vgaPEfV4TmpFlsoaotIVTmEfR2QqCLdk+MbRI5LYulV9Agbl8TSV9AjbFwSv7KGHUt25hhLHhP65KQsfqULjpYDUqzUEMQaiVhH90z1OXqmXVIfkpbGlwqmtdeIrb2Gau01YmvfRLf2uCT+yrEh7zqglHq+Ds0iftaXladZwoz4cUkaHTf6J2VpFvAzgaw8taQwrSIpqywsOK205DWk2ZnW164qHbcnrWSXrAzMXrSSXbYyMHvQSnbFysDsPSvZVSsDs+esZNesDMxes5KtWhmYPWYlu25lYPaWlexDKwOzp6xkH1kZmL1kJbthZWD2kJXsYysDs3esZJ9YGZg9YyX71MrA7BUr2U0rA7NHrGRrVgZmb1jJblkZmD1hJbttZWD2gpVs3crA7AEr2R0rA7P3q2R3rQzMnq+S3bMyMHu9SnbfysDs8SrZAysDs7erZA+tDMyerpI9sjJwe7k6N8PYDkZejtxrw7jcumwclmLjIFp/puXDI+bDbsOej7gNlxUzbzR7qVfy3ltPrmkPU1Kw30Z2mCEO5XxYvKJSs6uhLn/z1k7pFiD/i1iZEnKBX4nWM0kX1cw2sb6uoryk3AlYRREp9/DtooiYnt+VW0wenzv0Kfv+TUKE0PeJmsQYT1tw2Yj3ReaOlmQPdRojfVH3h6tvt+9q9bAhnwvQT8iYMRirOUloHhI0pwnNI0DTPD17Gj7toErWpSNWJsRadEv+1VoXZK3z0NNsVOZZcNN9GVuD8F4XyBefmecUWsQS0nWaTj/JiVHM7nuaMyHwMTvyac6UwMft0veljtjfi69kCCt9MCa05sxJwjFc8ZTPxZIv+I/aj3b5eRL2d6afg9pD11ECeTR3HyCeoFTP4E4yNQE9O6TOJhBPRir7l8zLMKAnhZIezACvnrFj2WKHJKreAXWR8yJZz2fns6rvhaRaPS1sdZiK6eTY6qrzMvzJiwpTFlDMvCoPTAzM4wE1EmxlYO9t4LbkKhlKhNh9cEUGxcL88Udt1fPlxVXD1LZWngfxuryF9sGMMkV7uXntFu3nKCWeHlNdZeyiBvL56V50f6DaY3Jr9GS791Ka83rg8RbeCbVbiBKERvJxNHqmZ5MTsIynFt0pQlfPbPP3pcag/sSpr+1DTzvrVTf4aWe9rgU/7RxEZRpY5ul3UHPpICrffM4UzRk7vcF54vIDQ3DlhHKv75hl784dE3TTd+COwfo0FscF7gI2tm0UTK82iXyZFPBlEvlio2B8Sd4xYGolmT6/l2m+R+Bj/Dc++whJL5SEokXJjkJp17iJIwUIEm2OBY9A+bb6pJls2up8Yz11/mz2ag4KRJ7ZKbFRKE+jHbBsj1XEuzg33ZuV4e8kx98ivU2cm/a3SP9TlZEkeFWUpH3GrUnQXMmMGWbObNKKl/w4NieGuJSSHwNXVvFxFFsGo5wyGJVQBqOcMrBxKWVwmePvZQn+Xub4a+NS/FXla49ZIwftqeJLvYg1TN5MzBSJFBMhcFy4x1fNsY2umDz5UZ6KjNFiRIyfT2lOI3LpmBOMhEYTtGHklQY0moorTH1yLGb3wmfQKqLIFVTW7lmF0FL3Tq6AMxAlq+tnpUDMaZaOGBsLlwvdmux7GXEZXAtOr+27bWDnU1lKciUFkxuqhWx/6toBKTZLzVrIjhIU67TZatYC3AOXWcrn1pyeF+gR88uUbosyfiXzlV7tc0Vx2fksbpuSb/VMQ7bFqPTieVOcbJtw8Wn+Dyz+D0ryf2Dx386PP3ttt7Ak91TbYVreKugS4kS7M3Ya7tn05Wnj8TXdJG2R1aPT/qATaM+s/cgZwqOR9MKcGS60sD1aWjvZGs6I2lAu3OttOqqKrdpBM5pxoRmxuZIscv04jfJaxjkFcJ6TV4xlWPTlvSHFLOLWLOOzOP1/kTXK+AzTzaP0i8k1zmz/WNYaq90erjbST42Vtdobr6FXsZocr7FyVpOz/tvncq+y1Nz1Vp5lM6/Jm6+Xwc2bic9fP4uRn4sFvFuMvLJRML4cRL4UW6WG6ht+NsHN13ktZxU8GyflrILje6ti5WEo2cgso4SqOSVULcHfao6/Ni7F38Ucf4u0LcPI+luktWnGhvQ36/OG4zrjTsG8KJvZ/FBt0vOJt4fn4vNB22/K+lvOfhO+Z5gU6hmSkfUq4ikZR68iejZQ/kMUU5fF7k+Aaq5YfZk7e4vsP9rviITvfnLfhWjPHcYv/fxH/N5ZlTJ/ieknQ9LMw8LMowzzqDAzPkK7844vz8MM05Z3PPMow7TlncJ093jmSR1oBaXG9LnDtdArvQIE37tSY2qPpoZeOQoiawHZmj61LUBb80NrmPt5Ggxzj5COs2Yq7iC+lmuk9LDWMLV9gChD4weWiF3b0SWCv+POSDYyujjNdd52dL+Sfw1uJPEzinM09RwVt3Fu/LQ9iIud92O9pTy52kZT26QyaKPLAHPits/0mnwvXF3vMvVJaPp0ZtwO7Q7DPM9KOeG/jSJSThVfRxGpcV5uLC6hiJhTf1y5xeTRlTdKjtYZ9llkWsljqRRfV9BUymft7KOptE+HwFIxZ4lAPmI8g0oPU2ZbAANz5s2WI3Ix+lDEYuIUqh1MnTyyXvc8Qui6Ziew/p7V9h6ou2bVXQN1q1bdKqi7EZ4/lb4q2AhPn4J07fsQWIJ7RoihmPlbx+FPXApLNOeZw9zk2ec0Om4dCW8B4zeeZk4JdI2zlLME46dQu5h5p1VjSraeuS6o5177Uj7bps6uUEzKZx7sZ/zcL+znfsbPfCbGT+3f/F5pb+b3wZR2F1mrXWK9dpE12yXWbdbf/FKg+buf46+Ni6vjrrOWcd4Zr4r4csXT9ScIz8L4wMrberqrqN9wP0ks7rhK3yedPBXb5Um+tjmtyqW7zFb5DGEzPB3PfA65KrllOXs4yaTvg6VZA8i1AmxN3iT5vEnwOZ9s9xlmrzlLY61AObvJtQJsQV5yer1kLQ+s5y5+raCFRT5/eyjrNI8efxfy1E2qZVguWvLOwSeA7WrYlyxGc2X9zJfo6W7L9PfAHGjKk+jThZMUkY6njGRObR6tEn1ysfwUzb3+IWhtptfH0n1xW8YRtNot9NXnnKsnBdtS5xn7MWLWO41sm/ntCatwbcjqQH7KRPqzz1V0mfcgz439+DMT7joQaX74uVzGcvxkZVUTaa/EEzOmVvTZcQu8rnqh9gJvuaIUO/x3U45LA/ZAnsCtZXVLEKnqlBfRqsz7O/z/A/YTaVmfNCFk35UpYtXz3Zj0qszVg5O3bt67W5FfC/YXe/fv3vvp3ftbP7v5wRJTX2+w73HebXaPvc8+YOu8Ve/ynDxnv2a/Zb97+beX/3r5n5efKdHXXwt13mGJr5ef/x/s41+A</latexit>
b<latexit sha1_base64="URFFgGflwoI4lopeshLNFmtz3XI=">AACLQnicvZ1JcyNJFYBzhm1olplhLkRwMdM09BBNh7qBGKIJIsar3O3utmx5H087JLksq1su1WixJXn0B/rEjQBOEMGB4GfAgT/AYX4CcCKGCC5zIJfKytoy33uq6rHDtpx673svM18ulVmVagbdzmBYqXz62utf+vJXvvq1N75+4xvf/Na333zr7e/sDXqjfsvbbfW6vf5BszHwuh3f2x12hl3vIOh7jYtm19tvvlgW7+9fev1Bp+fvDCeB99FFo+13zjqtxpAn7R03e93T5slbNyt3K/JrIfviXvjiJgu/ar23V99hx+yU9ViLjdgF85jPhvx1lzXYgH9/yO6xCgt42kfsmqf1+auOfN9jM3aD6464lMclGjz1Bf/d5v99GKb6/H/BHEjtFrfS5T99rrnAblX+Uflz5bPK3yt/qfyz8rmVdS0ZwpcJ/9tUul5w8ubL79b/B2pd8L9Ddm60nBpd/kr415E6FzKnd3LSBc2V9yE7Y7+Qee7wMghkiiiNlvLjcvqbz+oPtm9d/7Dyx8q/eTn8ofJp5a+8JPzL/7b+tOVt/57TBd/nWlfSpifte9z2Na8tj/M7nHXNVsLXvvStw6X8sF7ydQP+qsd/z7ifmiP8rIXpPZ4jHKkr/15kSI/D9IZTW/+f1t5JyLkILfk6S1gO07syhvs8alwUj41lpKQpq2F6EIv0fMKZrP1hhrAWpdt1dWSldRdzIu4Gu+UoC5HXTk4+lmPvCMJCxBDlp+rJ5xrXPF1TVYz0OGMm2/9H8l2fp3SkrOojRFqTyyyw22H09OR/6neyHhfYTc55j/+dIfzQMVqGH/lxTfNHR3oZ/pjWEffhmH/P44dfuiewF7rdlVEa2baaVypQpCgf8qwfy353zH8LH64TPrwn+2Is35Npysq5bF3C5x/w/5b4+2P56pLHmBoLxGhyP+wd4RLd4r3NSsju87GiK/Xfl6PuLPZqlhgVspwB7zP8kKPHvb7sH/Q77tzqfsOXPZ4iDsOxTbTjbg5Z6cyk/K9AC6KO23JcTfsp4ilLj8sf85F4hrbQ4HVCsSDkcRb68vWLRNQnmUZCtdkz/t5tGdfHskdvc8mhjD+XnaGcJ9lsqHdvh60FqteOnJvYaUaiiMcDWU8tSwTq947ZLwHOSLbjfjiaaY8/Cf35BK09SOib9GtJmrFnvI26WcJn8Z6qifshqSsl2rKkVH91h/++H8VGOzNnyCN3ov4EbmUddlf+nPKfWaTVifoXdw5O5Rw1vzUEMncBf/X92I9OhfucgL8aSXYA9qWnYb8SsB8BsrWIWgMlNbUGUgPZk6h+4WN2InuLLk89T81Vs5q+vDYKZG/RCCPrOpwfi979i/n+Of9214au3Sb/Xk7VuEi7lqnuvMblV3MZqyTG01zGUxJjO5exTWLUcxl1RIw35QzllE1lq+hJTjxdX5n2ZNutIMYQrdmLxlo7717EwxAXAdoiwbslgLVEYC0DrGUCawVgrRBYqwBrlcBaA1hrBFYVYFUJrHWAtU5gPQRYDwmsRwDrEYG1AbA2CKzHAOsxgfUEYD0hsJ4CrKcE1ibA2iSwagCrRmBtAawtAmsbYG0TWHWAVSewdgDWDoG1C7B2Caw9gLVHYO0DrH0C6wBgHRBYhwDrkMA6AlhHpJG7AdAaBM+aAKtJYLUAVovAOgVYpwQWNG/yCKwzgHVGYLUBVpvAOgdY5wRWB2B1CKznAOs5gfUCYL0gsLoAq0tgXQCsCwLLB1g+gQVdf/QIrABgBQTWxwDrYwKrD7D6BNYAYA0IrCHAGhJYI4A1IrAuAdYlgXUFsK4IrDHAGhNYE4A1IbCmAGtKGrmn4WpsMu0Zi682xFdt8V4Oo5V4iOvFPMbNNszKl6skkmtk+PmHJ9c0IbaRw4+uDaZ3l93suCR+fiLWXzGexyXxM5ZArtqKeyCgEeE4I42PGlzZT8llj+NSqWrXGUOOS+JnPT0U28jh5y0NuUcNs+OS+JlMQzLg64LjlCx+fgONQMehDH6WAxN9EnEMzlqPQxn8jAcmBiRiX+6mQEwthZ+1dMJ9SYgcl8S3uQairrQUfkZD6d/S0vgaPEfV4TmpFlsoaotIVTmEfR2QqCLdk+MbRI5LYulV9Agbl8TSV9AjbFwSv7KGHUt25hhLHhP65KQsfqULjpYDUqzUEMQaiVhH90z1OXqmXVIfkpbGlwqmtdeIrb2Gau01YmvfRLf2uCT+yrEh7zqglHq+Ds0iftaXladZwoz4cUkaHTf6J2VpFvAzgaw8taQwrSIpqywsOK205DWk2ZnW164qHbcnrWSXrAzMXrSSXbYyMHvQSnbFysDsPSvZVSsDs+esZNesDMxes5KtWhmYPWYlu25lYPaWlexDKwOzp6xkH1kZmL1kJbthZWD2kJXsYysDs3esZJ9YGZg9YyX71MrA7BUr2U0rA7NHrGRrVgZmb1jJblkZmD1hJbttZWD2gpVs3crA7AEr2R0rA7P3q2R3rQzMnq+S3bMyMHu9SnbfysDs8SrZAysDs7erZA+tDMyerpI9sjJwe7k6N8PYDkZejtxrw7jcumwclmLjIFp/puXDI+bDbsOej7gNlxUzbzR7qVfy3ltPrmkPU1Kw30Z2mCEO5XxYvKJSs6uhLn/z1k7pFiD/i1iZEnKBX4nWM0kX1cw2sb6uoryk3AlYRREp9/DtooiYnt+VW0wenzv0Kfv+TUKE0PeJmsQYT1tw2Yj3ReaOlmQPdRojfVH3h6tvt+9q9bAhnwvQT8iYMRirOUloHhI0pwnNI0DTPD17Gj7toErWpSNWJsRadEv+1VoXZK3z0NNsVOZZcNN9GVuD8F4XyBefmecUWsQS0nWaTj/JiVHM7nuaMyHwMTvyac6UwMft0veljtjfi69kCCt9MCa05sxJwjFc8ZTPxZIv+I/aj3b5eRL2d6afg9pD11ECeTR3HyCeoFTP4E4yNQE9O6TOJhBPRir7l8zLMKAnhZIezACvnrFj2WKHJKreAXWR8yJZz2fns6rvhaRaPS1sdZiK6eTY6qrzMvzJiwpTFlDMvCoPTAzM4wE1EmxlYO9t4LbkKhlKhNh9cEUGxcL88Udt1fPlxVXD1LZWngfxuryF9sGMMkV7uXntFu3nKCWeHlNdZeyiBvL56V50f6DaY3Jr9GS791Ka83rg8RbeCbVbiBKERvJxNHqmZ5MTsIynFt0pQlfPbPP3pcag/sSpr+1DTzvrVTf4aWe9rgU/7RxEZRpY5ul3UHPpICrffM4UzRk7vcF54vIDQ3DlhHKv75hl784dE3TTd+COwfo0FscF7gI2tm0UTK82iXyZFPBlEvlio2B8Sd4xYGolmT6/l2m+R+Bj/Dc++whJL5SEokXJjkJp17iJIwUIEm2OBY9A+bb6pJls2up8Yz11/mz2ag4KRJ7ZKbFRKE+jHbBsj1XEuzg33ZuV4e8kx98ivU2cm/a3SP9TlZEkeFWUpH3GrUnQXMmMGWbObNKKl/w4NieGuJSSHwNXVvFxFFsGo5wyGJVQBqOcMrBxKWVwmePvZQn+Xub4a+NS/FXla49ZIwftqeJLvYg1TN5MzBSJFBMhcFy4x1fNsY2umDz5UZ6KjNFiRIyfT2lOI3LpmBOMhEYTtGHklQY0moorTH1yLGb3wmfQKqLIFVTW7lmF0FL3Tq6AMxAlq+tnpUDMaZaOGBsLlwvdmux7GXEZXAtOr+27bWDnU1lKciUFkxuqhWx/6toBKTZLzVrIjhIU67TZatYC3AOXWcrn1pyeF+gR88uUbosyfiXzlV7tc0Vx2fksbpuSb/VMQ7bFqPTieVOcbJtw8Wn+Dyz+D0ryf2Dx386PP3ttt7Ak91TbYVreKugS4kS7M3Ya7tn05Wnj8TXdJG2R1aPT/qATaM+s/cgZwqOR9MKcGS60sD1aWjvZGs6I2lAu3OttOqqKrdpBM5pxoRmxuZIscv04jfJaxjkFcJ6TV4xlWPTlvSHFLOLWLOOzOP1/kTXK+AzTzaP0i8k1zmz/WNYaq90erjbST42Vtdobr6FXsZocr7FyVpOz/tvncq+y1Nz1Vp5lM6/Jm6+Xwc2bic9fP4uRn4sFvFuMvLJRML4cRL4UW6WG6ht+NsHN13ktZxU8GyflrILje6ti5WEo2cgso4SqOSVULcHfao6/Ni7F38Ucf4u0LcPI+luktWnGhvQ36/OG4zrjTsG8KJvZ/FBt0vOJt4fn4vNB22/K+lvOfhO+Z5gU6hmSkfUq4ikZR68iejZQ/kMUU5fF7k+Aaq5YfZk7e4vsP9rviITvfnLfhWjPHcYv/fxH/N5ZlTJ/ieknQ9LMw8LMowzzqDAzPkK7844vz8MM05Z3PPMow7TlncJ093jmSR1oBaXG9LnDtdArvQIE37tSY2qPpoZeOQoiawHZmj61LUBb80NrmPt5Ggxzj5COs2Yq7iC+lmuk9LDWMLV9gChD4weWiF3b0SWCv+POSDYyujjNdd52dL+Sfw1uJPEzinM09RwVt3Fu/LQ9iIud92O9pTy52kZT26QyaKPLAHPits/0mnwvXF3vMvVJaPp0ZtwO7Q7DPM9KOeG/jSJSThVfRxGpcV5uLC6hiJhTf1y5xeTRlTdKjtYZ9llkWsljqRRfV9BUymft7KOptE+HwFIxZ4lAPmI8g0oPU2ZbAANz5s2WI3Ix+lDEYuIUqh1MnTyyXvc8Qui6Ziew/p7V9h6ou2bVXQN1q1bdKqi7EZ4/lb4q2AhPn4J07fsQWIJ7RoihmPlbx+FPXApLNOeZw9zk2ec0Om4dCW8B4zeeZk4JdI2zlLME46dQu5h5p1VjSraeuS6o5177Uj7bps6uUEzKZx7sZ/zcL+znfsbPfCbGT+3f/F5pb+b3wZR2F1mrXWK9dpE12yXWbdbf/FKg+buf46+Ni6vjrrOWcd4Zr4r4csXT9ScIz8L4wMrberqrqN9wP0ks7rhK3yedPBXb5Um+tjmtyqW7zFb5DGEzPB3PfA65KrllOXs4yaTvg6VZA8i1AmxN3iT5vEnwOZ9s9xlmrzlLY61AObvJtQJsQV5yer1kLQ+s5y5+raCFRT5/eyjrNI8efxfy1E2qZVguWvLOwSeA7WrYlyxGc2X9zJfo6W7L9PfAHGjKk+jThZMUkY6njGRObR6tEn1ysfwUzb3+IWhtptfH0n1xW8YRtNot9NXnnKsnBdtS5xn7MWLWO41sm/ntCatwbcjqQH7KRPqzz1V0mfcgz439+DMT7joQaX74uVzGcvxkZVUTaa/EEzOmVvTZcQu8rnqh9gJvuaIUO/x3U45LA/ZAnsCtZXVLEKnqlBfRqsz7O/z/A/YTaVmfNCFk35UpYtXz3Zj0qszVg5O3bt67W5FfC/YXe/fv3vvp3ftbP7v5wRJTX2+w73HebXaPvc8+YOu8Ve/ynDxnv2a/Zb97+beX/3r5n5efKdHXXwt13mGJr5ef/x93y1+B</latexit>
CAM
····
····
min ⇧2U(a,b) log
LX
`=1
exp
✓ 1
⌘ h⇧, C`i
◆!
<latexit sha1_base64="adV/TDBqGDyqoqJaqG6T80xxsfE=">AACLyXicvZ1LcxvHEYBHeTrOw3J8SKpyga0ooVKKClSScsopV5lvSpREkODbsFgAuATXWi5WeJAAaJxyi35ADjklVTmk8jNyyR/IwT8hlVPKqcolh8zO7Ozsa6a7sZDJIgkOur/umel57MzuoBV4bn9QrX5+6ytf/drXv/HNN7715re/893vvXX77e8f9LvDXtvZb3e9bu+o1ew7nus7+wN34DlHQc9pXrY857D1YiV8//DK6fXdrr83GAfOJ5fNju+eu+3mgCed3n7VuHT905tGq+ud1dxKw/Ur4vX+gvjTvC/+tO5NKw2v22l4zvlgodEfXnIVx/M+XJw+f1JpOKMgeue812zfLE75m4PmtOE1/Y7nVCK4RK2choqNnnir0XM7F4N70Z/T23eqD6riq5J/sRi9uMOir1r37bV3WIOdsS5rsyG7ZA7z2YC/9liT9fn3x2yRVVnA0z5hNzytx1+54n2HTdmbXHfIpRwu0eSpL/jvDv/v4yjV5/+HzL7QbnMrHv/pcc0Ku1v9R/Uv1S+qf6/+tfrP6v+MrBvBCH0Z878tqesEp2/97of1/4Jal/zvgF1oLauGx1+F/rlC51Lk9H5Bekiz5X3AztmvRZ5dXgaBSAlLoy39uJr8/ov6B7t3b35S/VP1X7wc/lj9vPo3XhL+1X/af95xdv/A6SHf51rXwqYj7Dvc9g2vLYfzXc66YavRa1/45nIpP6qXYt2Av+ry31Pup+KEftai9C7PEY7kib+XOdKTKL1p1Vb/Z7X3UnI2Qlu8zhNWonRPxHCPR42N4rCRiJQsZS1KDxKRXkw4F7U/yBHW43SzroqsrO5SQcS9ye5ayiLMq1uQj5XEOyGhEjPC8pP15HONG56uqDJGupwxFe3/E/Guz1NcISv7iDCtxWUqbCGKnq74T/5O12OF3eGce/zvFOGHitF5+FEc1zR/VKTPwx/dOpI+NPj3LH74c/cE9kK1u3mURr6tFpUKFCnShyLrDdHvjvjv0IeblA/3RF+M5TsiTVq5EK0r9PnH/L9l/v5IvLriMSbHgnA0eRj1jnCJ7vDeZjVi9/hY4Qn998WoO028mqZGhTynz/sMP+Koca8n+gf1jj23qt/wRY8niYNobAvbsVdAljpTIf8haCGs444YV7N+hvGUpyflG3wknqItNHmdUCyE8jgLPfH6RSrq00wtIdvsOX9vQcR1Q/ToHS45EPFnszMQ8ySTDfnuQtRaoHp1xdzETNMSZTzui3pqGyJQvddgvwE4Q9GOe9Fopjz+LPLnM7R2P6Wv028Eacqe8zZqZ4U+h+/JmngYkTwh0RElJfur+/z3wzg2Ork5QxHZjfsTuJW57IH4OeM/01jLjfsXew7OxBy1uDUEIncBf/Vu4kelwn1OwF8NBTsA+9KzqF8J2E8B2VpMrYGSiloDqYHoSWS/8JKdit7C46kXmblqXtMX10aB6C2aUWTdRPPjsHf/cr5/xb/ttaFqt8W/VzI1HqbdiFR7XpPya4WMNRLjWSHjGYmxW8jYJTHqhYw6IsZbYoZyxiaiVXQFJ5murky7ou1WEWOI0uzGY62ZtxjzMMQlgLZE8G4ZYC0TWCsAa4XAWgVYqwTWGsBaI7DWAdY6gbUBsDYIrE2AtUlgPQJYjwisxwDrMYG1BbC2CKwnAOsJgfUUYD0lsJ4BrGcE1jbA2iawagCrRmDtAKwdAmsXYO0SWHWAVSew9gDWHoG1D7D2CawDgHVAYB0CrEMC6whgHRFYxwDrmMA6AVgnpJG7CdCaBM9aAKtFYLUBVpvAOgNYZwQWNG9yCKxzgHVOYHUAVofAugBYFwSWC7BcAutTgPUpgfUCYL0gsDyA5RFYlwDrksDyAZZPYEHXH10CKwBYAYH1EmC9JLB6AKtHYPUBVp/AGgCsAYE1BFhDAusKYF0RWNcA65rAGgGsEYE1BlhjAmsCsCakkXsSrcam056z5GpDctUW7+UgXomHuE7CY9xsQ6982UoivUaGn384Yk0TYms5/OjaZGp32c5OSuLnJ+H6K8bzpCR+xhKIVdvwHghoRGjkpPFRgyv7CbnscVwqVe46Y8hJSfysp4tiazn8vKUp9qhhdlISP5NpCgZ8XdDIyOLnN9AI1Ihk8LMcmOiTiCNw1tqIZPAzHpgYkIg9sZsCMZUUftbiRvuSEDkpiW9zTURdKSn8jIbSv2Wl8TV4garDC1IttlHUNpEqcwj72idRw3RHjG8QOSmJpW+gR9ikJJa+ih5hk5L4lTXsWLI3w1jyhNAnp2XxK11wtByRYqWGINZIxDq6Z6rP0DPtk/qQrDS+VDCtvUZs7TVUa68RW/s2urUnJfFXjk1x1wGl1It1aBbxs768PM0SZsRPStLouNE/LUuzgJ8J5OWpJYVpFWlZaaFitdIW15B6Z1pdu8p03J60lF02MjB70VJ2xcjA7EFL2VUjA7P3LGXXjAzMnrOUXTcyMHvNUnbDyMDsMUvZTSMDs7csZR8ZGZg9ZSn72MjA7CVL2S0jA7OHLGWfGBmYvWMp+9TIwOwZS9lnRgZmr1jKbhsZmD1iKVszMjB7w1J2x8jA7AlL2V0jA7MXLGXrRgZmD1jK7hkZmL1fKbtvZGD2fKXsgZGB2euVsodGBmaPV8oeGRmYvV0pe2xkYPZ0peyJkYHby1W5GSR2MIpyZF8bxuXWZuN4LjaO4vVnWj4cYj7MNsz5SNqwWdHzRr2Xei3uvXXEmvYgIwX7rWUHOeJAzIfDV1RqfjXU5m/R2indAuR/GSsTQi7wK9FqJmmj6tkm1tc1lJeUOwE3UETKPXz7KCKm57flFpPHTy36lH3/FiFC6PtELWKMZy3YbCT7In1HS7qHOkuQvqz7w+W33Xe5etgUzwWoJ2T0GIzVHKc0jwmak5TmCaCpn549i552kCVr0wlXJsK16Lb4q7QuyVoXkaf5qCyyYKf7Irb60b0ukC8+088ptIklpOo0m35aEKOY3fcsZ0zgY3bks5wJgY/bpe8JnXB/L7mSEVrpgTGhNKdWEo5hi6diLpZ8yX/kfrTNz9Oov9P9HNQePEsJFNHsfUD4BKV8Bnecqwno2SF5NkH4ZKS0f8WcHAN6UijtwRTw6jlriBY7IFHVDqiNXBTJaj47m1V1LyTV6llpq4NMTKfHVludz8OfoqjQZQHFzOvyQMfALB5QI8FUBubeBm5LtpKhRIjZB1tkUCzMHn/UVj1bXmw1TG1r8/MgWZd30T7oUaZsLzer3bL9HKXEs2OqrYxt1EA8P92N7w+Ue0x2ja5o905Gc1YPHN7C3Ui7jShBaCQfxaNndjY5Bst4YtCdIHTVzLZ4X2oE6o+t+so+9LSzWnWDn3ZW61rw085BXKaBYZ5+HzWXDuLyLeZM0JyR1RucJzY/MARbTij3+o5Y/u7cEUE3ewfuCKxPbXFU4i5gbdtEwfRq49iXcQlfxrEvJgrGl/QdA7pW0umze5nlOwQ+xn/ts4+QdCJJKFqk7DCSto2bOFKAINHmWPAIVGyrR5rJZq3ONtZT5896r+aoROTpnRIThfI02hHL91hlvEtys73ZPPwdF/hbprdJcrP+lul/NkQkhbwNlKR5xq1I0FxJjxl6zqzTypf8KDEnhriUkh8BV1bJcRRbBsOCMhjOoQyGBWVg4lLK4KrA36s5+HtV4K+JS/FXlq85ZrUctKeKL/Uy1jB50zFTJlJ0hMBxYR9fFcc0umLy5Md5KjNGhyNi8nxKfRqRTUefYBRqtEAbWl5qQKNpeIWpTo7F7F74DFpFDHMFlbV9VhFqyXsnV8EZiJRV9bNaIuYUS0WMiYXLhWpN5r2MpAyuBWfX9u02sPOpPCW9koLJDdVCvj+17YCUm6XmLeRHCYp12mw1bwHugedZyhfGnF6U6BGLy5RuizJ+pfOVXe2zRfG881neNiXf8pmGfIuR6eXzJjn5NmHj0/zvG/zvz8n/vsF/Mz/57LXZwrLYU+1EaUWroMuIE+3O2Vm0Z9MTp40n13TTtCVWj0/7g06gPTf2I+cIj4bCC31meKiF7dGy2unWcE7UhnJhX29TUVVu1Q6a0YxKzYj1lWSZ68dJnNd5nFMA5zl9xTgPi764N6ScRdyaZXIWp/4vs0aZnGHaeZR+Mb3Gme8f57XGaraHq43sU2PzWu1N1tDrWE1O1th8VpPz/pvncq+z1Oz1Nj/Lel5TNF+fB7doJj57/SzFfi6V8G4p9spEwfhyFPtSbpUaqm/42QQ7X+V1Pqvg+TiZzyo4vrcqVx6ako/MeZTQRkEJbczB340Cf01cir9LBf6WaVuakfe3TGtTjC3hb97nLct1xv2SeZE28/mh2qTnE28Pz8Xng7bflPd3PvtN+J5hXKpnSEfW64indBy9jujZQvkPUXRdlrs/Aaq5cvWl7+wts/9oviMSvvvJfheiOXcYv9TzH8l7Z2XK7CWmngzJMo9LM09yzJPSzOQIbc87vjyPc0xT3vHMkxzTlHcK097j6Sd1oBWUGlPnDtcir9QKEHzvSo3JPZoaeuUoiK0FZGvq1LYAbc2PrGHu52kyzD1CKs5ambiD+EqumdHDWsPU9hGiDLUfWCJ2bUeVCP6OOy3ZzOniNDd521H9SvE1uJbEzygu0NQLVNwmucnT9iAudt6P9Zby5GoHTe2QyqCDLgPMids+U2vy3Wh13WPyk9DU6cy4Hdo9hnmelXLCfwdFpJwqvokiUuN8vrG4jCJiTv2x5RaTR1veKDnaZNhnkWklj6VSfF1FUymftXOIptI+HQJLxZwlAvmI8QwqPUyZ7QAMzJk3O5bIxehDEYuJU6h2MHXy2Hjd8xiha5udwPoHRtsHoO66UXcd1N0w6m6AulvR+VPZq4Kt6PQpSNe8D4El2GeEGIqev7kWf5JSWKI+zxzmps8+p9Fx60h4Cxi/8TR9SqBtnKWcJZg8hdrGLDqtGlOy9dx1Qb3w2pfy2TZ1do1iUj7z4DDn52FpPw9zfhYzMX4q/2b3Snkzuw+6tD1krXrEevWQNesR6zbvb3Ep0Pw9LPDXxMXVsWetZZx32qsyvlzzdPUJwtMoPrDypp7uOu437E8Sh3dcZe+TTp+KbfOkWFufVmXTXWFrfIawHZ2Opz+HXJbcipg9nObSD8HSrAHkWgm2Im+TfN4m+FxMNvsMs9etpbFeopzt5FoJdkhetnq9bCwPrOc2fq2khSU+f3sk6rSInnwX8tROquVYNlr6zsGngO2NqC9ZiufK6pmvsKdbEOn3wBwoytP404XTlDAdTxmKnJo8WiP6ZGP5GZp9/SOkdZhaH8v2xR0RR9Bqd6gvP+dcPinYETrP2c8Qs95JbFvPb09ZlWtDVvviUyayn30uo0u/B3mu7SefmbDXQZjmR5/LpS0nT1aWNZH1KnxiRteKOjuuwuuqG2lXeMsNS9Hlv1tiXOqzD8QJ3EpWtYQwVZ7yErYq/f4e//+I/VxYVidNhLLviZRw1fO9hPSayNUHp7fvLD6oiq+K+cXBwweLv3jwcOeXdz5aZvLrDfYjzltgi+x99hHb5K16n+fk37d+cKty691XW69evhq9mkjRr9yKdN5hqa9Xv/0/d1uOqg==</latexit>
Figure 2: Semantic correspondence framework based on FROT.
A potential problem with FROT is that the estimation depends significantly on the magnitude of the cost of each layer (also known as a group). Hence, normalizing each cost matrix is important. Therefore, we normalized each feature vector by f (`,s)i ← f (`,s) i /‖f (`,s) i ‖2. Consequently, the cost matrix is given by [C`]ij = 2 − 2f
(`,s) i > f (`,s′) j . We can use dis-
tances such as the L1 distance.

heading: Computation of a and b with
text: staircase re-weighting: For semantic correspondence, setting a ∈ Rhsws and b ∈ Rhs′ws′ is important because semantic correspondence can be affected by background clutter. Therefore, we generated the class activation maps (Zhou et al., 2016) for the source and target images and used them as a and b, respectively. For CAM, we chose the class with the highest classification probability and normalized it to the range [0, 1].

heading: 4 RELATED WORK
text: OT algorithms: The Wasserstein distance can be determined by solving the OT problem. An advantage of the Wasserstein distance is its robustness to noise; moreover, we can obtain the transport
plan, which is useful for many machine learning applications. To reduce the computation cost for the Wasserstein distance, the sliced Wasserstein distance is useful (Kolouri et al., 2016). Recently, a tree variant of the Wasserstein distance was proposed (Evans & Matsen, 2012; Le et al., 2019; Sato et al., 2020); the sliced Wasserstein distance is a special case of this algorithm.
The approach most closely related to FROT is a robust variant of the Wasserstein distance, called the subspace robust Wasserstein distance (SRW) (Paty & Cuturi, 2019). SRW computes the OT problem in a discriminative subspace; this is possible by solving dimensionality-reduction problems. Owing to the robustness, SRW can successfully compute the Wasserstein distance from noisy data. The max–sliced Wasserstein distance (Deshpande et al., 2019) and its generalized counterpart (Kolouri et al., 2019) can also be regarded as subspace-robust Wasserstein methods. Note that SRW (Paty & Cuturi, 2019) is a min–max based approach, while the max–sliced Wasserstein distances (Deshpande et al., 2019; Kolouri et al., 2019) are max–min approaches. The FROT is a feature selection variant of the Wasserstein distance, whereas the subspace approaches are used for dimensionality reduction.
As a parallel work, a general minimax optimal transport problem called the robust Kantorovich problem (RKP) was recently proposed (Dhouib et al., 2020). RKP involves using a cutting-set method for a general minmax optimal transport problem that includes the FROT problem as a special case. The approaches are technically similar; however, our problem and that of Dhouib et al. (2020) are intrinsically different. Specifically, we aim to solve a high-dimensional OT problem using feature selection and apply it to semantic correspondence problems, while the RKP approach focuses on providing a general framework and uses it for color transformation problems. As a technical difference, the cutting-set method may not converge to an optimal solution if we use the regularized OT (Dhouib et al., 2020). By contrast, because we use a Frank–Wolfe algorithm, our algorithm converges to a true objective function with regularized OT solvers. The multiobjective optimal transport (MOT) is an approach (Scetbon et al., 2020) parallel to ours. The key difference between FROT and MOT is that MOT tries to use the weighted sum of cost functions, while FROT considers the worst case. Moreover, as applications, we focus on the cost matrices computed from subsets of features, while MOT considers cost matrices with different distance functions.

heading: 5 EXPERIMENTS

heading: 5.1 SYNTHETIC DATA
text: We compare FROT with a standard OT using synthetic datasets. In these experiments, we initially generate two-dimensional vectors x ∼ N(µx,Σx) and y ∼ N(µy,Σy). Here, we set µx = (−5, 0)>, µy = (5, 0)>, Σx = Σy = ((5, 1)>, (4, 1)>). Then, we concatenate zx ∼ N(08, I8) and zy ∼ N(08, I8) to x and y, respectively, to give x̃ = (x>, z>x ), ỹ = (y>, z>y ). For FROT, we set η = 1.0 and the number of iterations of the Frank–Wolfe algorithm as T = 10. The regularization parameter is set to = 0.02 for all methods. To show the proof-of-concept, we set the true features as a group and the remaining noise features as another group.
Fig. 1a shows the correspondence from x and y with the vanilla OT algorithm. Figs. 1b and 1c show the correspondence of FROT and OT with x̃ and ỹ, respectively. Although FROT can identify
a suitable matching, the OT fails to obtain a significant correspondence. We observed that the α parameter corresponding to a true group is α1 = 0.9999. Moreover, we compared the objective scores of the FROT with LP, FW-EMD, and FW-Sinkhorn ( = 0.1). Figure 3a shows the objective scores of FROTs with the different solvers, and both FW-EMD and FW-Sinkhorn can achieve almost the same objective score with a relatively small η. Moreover, Figure 3b shows the mean squared error between the LP method and the FW counterparts. Similar to the objective score cases, it can yield a similar transport plan with a relatively small η. Finally, we evaluated the FW-Sinkhorn by changing the regularization parameter η. In this experiment, we set η = 1 and varied the values. The result shows that we can obtain an accurate transport plan with a relatively small .

heading: 5.2 SEMANTIC CORRESPONDENCE
text: We evaluated our FROT algorithm for semantic correspondence. In this study, we used the SPair71k (Min et al., 2019b). The SPair-71k dataset consists of 70, 958 image pairs with variations in viewpoint and scale. For evaluation, we employed a percentage of accurate key points (PCK), which counts the number of accurately predicted key points given a fixed threshold (Min et al., 2019b). All semantic correspondence experiments were run on a Linux server with NVIDIA P100.
For the optimal transport based frameworks, we employed ResNet101 (He et al., 2016) pretrained on ImageNet (Deng et al., 2009) for feature and activation map extraction. The ResNet101 consists of 34 convolutional layers and the entire number of features is d = 32, 576. Note that we did not fine-tune the network. We compared the proposed method with several baselines (Min et al., 2019b) and the SRW1. Owing to the computational cost and the required memory size for SRW, we used the first and the last few convolutional layers of ResNet101 as the input of SRW. In our experiments, we empirically set T = 3 and = 0.1 for FROT and SRW, respectively. For SRW, we set the number of latent dimension as k = 50 for all experiments. HPF (Min et al., 2019a) and OT-HPF (Liu et al., 2020) are state-of-the-art methods for semantic correspondence. HPF and OT-HPF required the validation dataset to select important layers, whereas SRW and FROT did not require the validation dataset. OT is a simple optimal transport-based method that does not select layers.
Table 1 lists the per-class PCK results obtained using the SPair-71k dataset. FROT (η = 0.3) outperforms most existing baselines, including HPF and OT. Moreover, FROT (η = 0.3) is consistent with OT-HPF (Liu et al., 2020), which requires the validation dataset to select important layers. In this experiment, setting η < 1 results in favorable performance (See Table 3 in the Appendix). The computational costs of FROT is 0.29, while SRWs are 8.73, 11.73, 15.76, respectively. Surprisingly, FROT outperformed SRWs. However, this is mainly due to the used input layers. Therefore, scaling up SRW would be an interesting future work.
We further evaluated FROT by tuning hyperparameters η and using validation sets, where the maximum search ranges for η and are set to 0.2 to 2.0 and 0.1 to 0.6 with intervals of 0.1, respectively. Figure 6 in Appendix shows the average PCK scores for (η, ) pairs on the validation split of SPair71k. By using hyperparameter search, we selected (η = 0.2, = 0.4) as an optimal parameter. The FROT with optimal parameters outperforms the state-of-the-art method (Liu et al., 2020).
1https://github.com/francoispierrepaty/SubspaceRobustWasserstein

heading: 5.3 FEATURE SELECTION EXPERIMENTS
text: Since FROT finds the transport plan and discriminative features between X and Y , we can use FROT as a feature-selection method. We considered X ∈ Rd×n and Y ∈ Rd×m as sets of samples from classes 1 and 2, respectively. The optimal important feature is given by
α̂` = exp
( 1 η 〈Π̂,C`〉 )
∑d `′=1 exp ( 1 η 〈Π̂,C`′〉 ) , with Π̂ = argmin Π∈U(µ,ν) η log
( d∑
`=1
exp
( 1
η 〈Π,C`〉
)) ,
where [C`]ij = (x (`) i − y (`) j ) 2. Finally, we selected the top K features by the ranking α̂. Hence, α changes to a one-hot vector for a small η and to αk ≈ 1L for a large η. Here, we compared FROT with several baseline algorithms in terms of solving feature-selection problems. In this study, we employed a high-dimensional and a few sample datasets with two class classification tasks (see Table 2). All feature selection experiments were run on a Linux server with an Intel Xeon CPU E7-8890 v4 with 2.20 GHz and 2 TB RAM.
In our experiments, we initially randomly split the data into two sets (75% for training and 25% for testing) and used the training set for feature selection and building a classifier. Note that we standardized each feature using the training set. Then, we used the remaining set for the test. The trial was repeated 50 times, and we considered the averaged classification accuracy for all trials. Considered as baseline methods, we computed the Wasserstein distance, maximum mean discrepancy (MMD) (Gretton et al., 2007), and linear correlation2 for each dimension and sorted them in descending order. Note that the Wasserstein distance is computed via sorting, which is computationally more efficient than the Sinkhorn algorithm when d = 1. Then, we selected the top K features as important features. For FROT, we computed the feature importance and selected the features that had significant importance scores. In our experiments, we set η = 1.0 and T = 10. Then, we trained a two-class SVM3 with the selected features.
Fig. 4 shows the average classification accuracy relative to the number of selected features. From Figure 4, FROT is consistent with the Wasserstein distance-based feature selection and outperforms the linear correlation method and the MMD for two datasets. Table 2 shows the computational time(s) of the methods. FROT is about two orders of magnitude faster than the Wasserstein distance and is also faster than MMD. Note that although MMD is as fast as the proposed method, it cannot determine the correspondence between samples.

heading: 6 CONCLUSION
text: In this paper, we proposed FROT for high-dimensional data. This approach jointly solves feature selection and OT problems. An advantage of FROT is that it is a convex optimization problem and can determine an accurate globally optimal solution using the Frank–Wolfe algorithm. We used FROT for high-dimensional feature selection and semantic correspondence problems. Through extensive experiments, we demonstrated that the proposed algorithm is consistent with state-of-theart algorithms in both feature selection and semantic correspondence.
2https://scikit-learn.org/stable/modules/feature_selection.html 3https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html

heading: APPENDIX

heading: RELATED WORK
text: In addition to accelerating the computation, structured optimal transport incorporates structural information directly into OT problems (Alvarez-Melis et al., 2018). Specifically, they formulate the submodular optimal transport problem and solve the problem using a saddle-point mirror prox algorithm. Recently, more complex structured information was introduced in the OT problem, including the hierarchical structure (Alvarez-Melis et al., 2020; Yurochkin et al., 2019). These approaches successfully incorporate structured information into OT problems with respect to data samples. By contrast, FROT incorporates the structured information into features.
OT applications: OT has received significant attention for use in several computer vision tasks. Applications include Wasserstein distance estimation (Peyré et al., 2019), domain adaptation (Yan et al., 2018), multitask learning (Janati et al., 2019), barycenter estimation (Cuturi & Doucet, 2014), semantic correspondence (Liu et al., 2020), feature matching (Sarlin et al., 2019), photo album summarization (Liu et al., 2019), generative model (Arjovsky et al., 2017; Bunne et al., 2019), graph matching (Xu et al., 2019a;b), and the semantic correspondence (Liu et al., 2020).

heading: PROOF OF PROPOSITION 1
text: For the distance function d(x,y), we prove that
FRWDp(µ, ν) =   min
Π∈U(µ,ν) max α∈ΣL
n∑
i=1
m∑
j=1
πij
L∑
`=1
α`d(x (`) i ,y (`) j ) p
  1/p
is a distance for p ≥ 1. The symmetry can be read directly from the definition as we used distances that are symmetric. For the identity of indiscernibles, when FRWDp(µ, ν) = 0 with the optimal α and Π, there exists ` such that α` > 0 (as α is in the simplex set). As there is a max in the definition and∑ ij πijα`d(x (`) i ,y (`) j ) p = 0, this means that ∀`,∑ij πijd(x (`) i ,y (`) j )
p = 0 and ∀`, µ(`) = ν(`). Therefore, we have µ = ν when FRWDp(µ, ν) = 0.
When µ = ν, this means that xi = yi, ai = bi,∀ i, and n = m, and we have d(xi,yj) = 0 for i = j. Thus, for any α` ≥ 0, the optimal transport plan is πii > 0 for d(xi,yj) = 0 and πij = 0 for d(xi,yj) > 0. Therefore, when µ = ν, we have FRWDp(µ, ν) = 0.
TRIANGLE INEQUALITY
Let µ = ∑n i=1 aiδxi , ν = ∑m j=1 bjδyj , γ = ∑u k=1 ckδzk and α ∈ ΣL, we prove that
FRWDp(µ, γ) ≤ FRWDp(µ, ν) + FRWDp(ν, γ).
To simplify the notations in this proof, we define D` as the distance ”matrix” such that [D`]ij = d(x
(`) i ,y (`) j ) is the ith-row and jth-column element of the matrix D`, [D`]jk = d(y (`) j , z (`) k ), and
[D`]ik = d(x (`) i , z (`) k ). Moreover, note that D p ` is the ”matrix,” where each element is an element ofD` raised to the power p.
Consider that P ∈ U(µ, ν) is the optimal transport plan of FRWDp(µ, ν), and Q ∈ U(ν, γ) is the optimal transport plan of FRWDp(ν, γ), where γ = ∑r k=1 ciδzi is a discrete measure. Similar to the proof for the Wasserstein distance in (Peyré et al., 2019), let S = P diag(1/b̃)Q with b̃ be a vector such that b̃j = bj if bj > 0, and bj = 1 otherwise. We can show that S ∈ U(µ, γ).
( min
R∈U(µ,γ)
L∑
`=1
α`〈R,Dp` 〉 ) 1 p ≤ ( L∑
`=1
α`〈S,Dp` 〉 ) 1 p = ( L∑
`=1
α` ∑
ik
Sik[D`] p ik
) 1 p
≤
  L∑
`=1
α` ∑
ik
[D`] p ik
∑
j
pijqjk
b̃j
  1 p
=
  L∑
`=1
α` ∑
ijk
[D`] p ik
pijqjk
b̃j
  1 p
≤
  L∑
`=1
α` ∑
ijk
([D`]ij + [D`]jk) p pijqjk
b̃j
  1 p
By letting gijk` = [D`]ij(α`pijqjk/b̃j)1/p and hijk` = [D`]ij(α`pijqjk/b̃j)1/p, the right-hand side of this inequality can be rewritten as
  L∑
`=1
α` ∑
ijk
([D`]ij + [D`]jk) p pijqjk
b̃j
  1 p
=
  L∑
`=1
∑
ijk
(gijk` + hijk`) p
  1 p
≤
  L∑
`=1
∑
ijk
gpijk`
  1 p
+
  L∑
`=1
∑
ijk
hpijk`
  1 p
≤
  L∑
`=1
α` ∑
ijk
[D`] p ij
pijqjk
b̃j
  1 p
+
  L∑
`=1
α` ∑
ijk
[D`] p jk
pijqjk
b̃j
  1 p
by the Minkovski inequality.
( min
R∈U(µ,γ)
L∑
`=1
α`〈R,Dp` 〉 ) 1 p ≤   L∑
`=1
α` ∑
ij
[D`] p ijpij
∑
k
qjk
b̃j
  1 p
+
  L∑
`=1
α` ∑
ik
[D`] p jkqjk
∑
j
pij
b̃j
  1 p
≤
  L∑
`=1
α` ∑
ij
[D`] p ijpij
  1 p
+
( L∑
`=1
α` ∑
ik
[D`] p jkqjk
) 1 p
≤  max α∈ΣL L∑
`=1
α` ∑
ij
[D`] p ijpij
  1 p
+ ( max α∈ΣL L∑
`=1
α` ∑
ik
[D`] p jkqjk
) 1 p
≤ FRWDp(µ, ν) + FRWDp(ν, γ)
This inequality is valid for all α. Therefore, we have
FRWDp(µ, ν) ≤ FRWDp(µ, ν) + FRWDp(ν, γ)

heading: FROT WITH LINEAR PROGRAMMING
text: Linear Programming: The FROT is a convex piecewise-linear minimization because the objective is the max of linear functions. Thus, we can solve the FROT problem via linear programming:
min Π∈U(µ,ν),t
t, s.t. 〈Π,C`〉 ≤ t, ` = 1, 2, . . . , L.
This optimization can be easily solved using an off-the-shelf LP package. However, the computational cost of this LP problem is high in general (i.e., O(n3), n = m).
The FROT problem can be written as
min Π max `∈{1,2,...,L}
〈Π,C`〉,
s.t. Π1m = a,Π>1n = b,Π ≥ 0.
This problem can be transformed to an equivalent linear program by first forming an epigraph problem:
min Π,t t,
s.t. max `∈{1,2,...,L} 〈Π,C`〉 ≤ t
Π1m = a,Π >1n = b,Π ≥ 0.
Thus, the linear programming for FROT is given as
min Π,t t
s.t. 〈Π,C`〉 ≤ t, ` = 1, 2, . . . , L Π1m = a,Π >1n = b,Π ≥ 0.
Next, we transform this linear programming problem into the canonical form. For matrix Π = (π1 π2 . . . πn)
> ∈ Rn×m and πi ∈ Rn, we can vectorize the matrix using the following linewise operator:
vec(Π) = (π>1 π > 2 . . . π > n ) > ∈ Rnm.
Using this vectorization operator, we can write 〈Π,C`〉 ≤ t as  vec(C1)> −1 vec(C2)> −1 ... ...
vec(CL)> −1
  ( vec(Π) t ) ≤ 0L,
where 0L ∈ RL is a vector whose elements are zero. For the constraints Π1m = a and Π>1n = b, we can define vectors q1, . . . , qn ∈ Rnm and r1, . . . , rm ∈ Rnm such that q>i vec(Π) = ai and r>j vec(Π) = bj in this way:
q1 = (1 > m,0 > m, . . . ,0 > m) >, q2 = (0 > m,1 > m, . . . ,0 > m) >,
...
qn = (0 > m,0 > m, . . . ,1 > m) >
and
r1 = (1,0 > m−1, 1,0 > m−1, . . . , 1,0 > m−1) >, r2 = (0, 1,0 > m−1, 1,0 > m−1, . . . , 1,0 > m−2) >,
...
rm = (0 > m−1, 1,0 > m−1, 1, . . .0 > m−1, 1) >
We can collect these vectors to obtain the vectorized constraints:  q>1 0 q>2 0 ... ...
q>n 0
  ( vec(Π) t ) = a,   r>1 0 r>2 0 ...
... r>m 0
  ( vec(Π) t ) = b,
Thus, we can rewrite the linear programming as
min u
e>u
s.t. (A> − 1L)u ≤ 0L, (Q> 0n)u = a, (R> 0m)u = b,u ≥ 0, where u = (vec(Π)> t)> ∈ Rnm+1, e = (0>nm 1)> ∈ Rnm+1 is the unit vector whose nm + 1- th element is 1, A = (vec(C1), . . . , vec(CL)) ∈ Rnm×L, Q = (q1, . . . , qn) ∈ Rnm×n, and R = (r1, . . . , rm) ∈ Rnm×m. Q = (In In . . . In)

heading: PROOF OF LEMMA 2
text: We optimize the function with respect to α:
max α J(α)
s.t. α>1K = 1, α1, . . . , αK ≥ 0, where
J(α) =
L∑
`=1
α`φ` − η L∑
`=1
α`(logα` − 1). (8)
Because the entropic regularization is a strong convex function and its negative counterpart is a strong concave function, the maximization problem is a concave optimization problem.
We consider the following objective function with the Lagrange multiplier :
J̃(α) =
L∑
`=1
α`φ` − η L∑
`=1
α`(logα` − 1) + (α>1K − 1)
Note that owing to the entropic regularization, the nonnegative constraint is automatically satisfied.
Taking the derivative with respect to α`, we have
∂J̃(α)
∂α` = φ` − η
( logα` − 1 + α` 1
α`
) + = 0.
Thus, the optimal α` has the form
α` = exp
( 1
η φ`
) exp (
η
) .
α` satisfies the sum to one constraint.
exp
(
η
) =
1 ∑L `′=1 exp ( 1 ηφ`′ )
Hence, the optimal α` is given by
α` = exp
( 1 ηφ` )
∑L `′=1 exp ( 1 ηφ`′ ) .
Substituting this into Eq.(8), we have
J(α∗) = L∑
`=1
exp (
1 ηφ`
)
∑L `′=1 exp ( 1 ηφ`′
)φ` − η L∑
`=1
exp (
1 ηφ`
)
∑L `′=1 exp ( 1 ηφ`′ )
 log  
exp (
1 ηφ`
)
∑L `′=1 exp ( 1 ηφ`′ )
 − 1  
= η log
( L∑
`=1
exp
( 1
η φ`
)) + η
Therefore, the final objective function is given by
J(α∗) = η log
( L∑
`=1
exp
( 1
η φ`
)) + η

heading: PROOF OF PROPOSITION 3
text: Proof: For 0 ≤ θ ≤ 1 and η > 0, we have L∑
`=1
exp
( 1
η 〈θΠ1 + (1− θ)Π2,D`〉
) = L∑
`=1
exp
( θ
η 〈Π1,D`〉+ (1− θ) η 〈Π2,D`〉
)
=
L∑
`=1
exp
( 1
η 〈Π1,D`〉
)θ exp ( 1
η 〈Π2,D`〉
)1−θ
≤ ( L∑
`=1
exp
( 1
η 〈Π1,D`〉
))θ ( L∑
`=1
exp
( 1
η 〈Π2,D`〉
))1−θ
Here, we use Hölder’s inequality with p = 1/θ, q = 1/(1− θ), and 1/p+ 1/q = 1. Applying a logarithm on both sides of the equation and then premultiplying η, we have
η log
( L∑
`=1
exp
( 1
η 〈θΠ1 + (1− θ)Π2,D`〉
)) ≤ θη log ( L∑
`=1
exp
( 1
η 〈Π1,D`〉
))
+ (1− θ)η log ( L∑
`=1
exp
( 1
η 〈Π2,D`〉
))

heading: PROOF OF PROPOSITION 4
text: Theorem 5 (Jaggi, 2013) For each t ≥ 1, the iterates Π(t) of Algorithms 1, 2, 3, and 4 in (Jaggi, 2013) satisfy
f(Π(t))− f(Π∗) ≤ 2Cf t+ 2 (1 + δ),
where Π∗ ∈ D is an optimal solution to problem Π∗ = argmin
Π∈D f(Π),

heading: Cf is the curvature constant defined as
text: Cf := sup Π,Π̂,γ
2
γ2 (f(Π′)− f(Π)− 〈Π′ −Π,∇f(Π)〉
s.t. Π, Π̂ ∈ D, γ ∈ [0, 1],Π′ = Π + γ(Π̂−Π), δ ≥ 0 is the accuracy with which internal linear subproblems are solved.
Lemma 6 (Jaggi, 2013) Let f be a convex and differentiable function with its gradient ∇f being Lipschitz-continuous w.r.t. some norm ‖ · ‖ over the domain D with Lipschitz-constant L > 0. Then,
Cf ≤ diam‖·‖(D)2L
Definition 7 The softmax function is given by
σ(z) = 1
∑L `′=1 exp(λz`′)
  exp(λz1) exp(λz2)
... exp(λzL)
  ,
where λ > 0 is referred to as the inverse temperature constant.
Lemma 8 (Gao & Pavel, 2017) The softmax function σ(·) is L-Lipschitz with respect to ‖ · ‖2 with L = λ, that is for all z, z′ ∈ Rn,
‖σ(z)− σ(z′)‖2 ≤ λ‖z − z′‖2, where λ is the inverse temperature constant.
The derivative of Gη(Π) is given as
∂Gη(Π)
∂Π =
L∑
`=1
exp (
1 η 〈Π,C`〉
)
∑L `′=1 exp ( 1 η 〈Π,C`′〉 )C` =MΠ.
Thus, we have
vec(MΠ) = ΦpΠ,
where
Φ = (vec(C1), vec(C2), . . . , vec(CL)) ∈ Rnm×L,
pΠ =
 
exp (
1 η 〈Π,C1〉
)
∑L `′=1 exp ( 1 η 〈Π,C`′〉
) , . . . , exp
( 1 η 〈Π,CL〉 )
∑L `′=1 exp ( 1 η 〈Π,C`′〉 )
  >
∈ RL.
Here, pΠ is the softmax function σ(z) with z` = 〈Π,C`〉. We have
‖∇Gη(Π)−∇Gη(Π′)‖2 = ‖ΦpΠ −ΦpΠ′‖2 ≤ ‖Φ‖op‖pΠ − pΠ′‖2,
≤ 1 η ‖Φ‖op‖Φ>vec(Π)−Φ>vec(Π′)‖2 (Lemma 8 with λ = 1 η )
≤ 1 η ‖Φ‖op‖Φ>‖op‖vec(Π)− vec(Π′)‖2
where ‖ · ‖op is the operator norm. We have ‖Φ‖op = ‖Φ>‖op, ‖Φ>Φ‖op = ‖Φ‖2op, and ‖vec(Π) − vec(Π′)‖2 ≤ √ 2. Therefore, the Lipschitz constant for the gradient is L = 1η‖Φ‖2op = 1 ησmax(Φ
>Φ), and the curvature constant is bounded above by Cf ≤ 2L, where σmax(Φ>Φ) is the largest eigenvalue of the matrix Φ>Φ. By plugging Cf in Theorem 5, we have
Gη(Π (t))−Gη(Π∗) ≤
4σmax(Φ >Φ)
η(t+ 2) (1 + δ).

heading: MAX/MIN FORMULATION
text: We define the max–min formulation of the FROT as
max α∈ΣL
L∑
`=1
α` min Π∈U(a`,b`)
n∑
i=1
m∑
j=1
πijc(x (`) i ,y (`) j ),
where ΣL = {α ∈ RL+ : α>1L = 1} is the probability simplex, the set of probability vectors in RL.
This problem can be solved by computing the group that maximizes the optimal transport distance k∗ = argmax kW1(µ (`), ν(`)) and then by considering α∗ = δk∗ as a one-hot vector.
The result of this formulation provides an intuitive idea (the same as for the robust Wasserstein method). Hence, we maximize the group (instead of the subspace) that provides the optimal result. However, the formulation requires solving the OT problem L times. This approach may not be suitable if we have a large L. Moreover, the argmax function is generally not differentiable.
Relation to the max-sliced Wasserstein distance: The max-sliced Wasserstein-2 distance can be defined as (Deshpande et al., 2019)
max-W2(µ, ν) =  max w∈Ω min Π∈U(a`,b`) n∑
i=1
m∑
j=1
πij(w >xi −w>yj)2
  1 2
,
where Ω ⊂ Rd is the set of all possible directions on the unit sphere. The max–sliced Wasserstein is a max–min approach. That is, for each w, it requires solving the OT problem. The max–min approach is suited for simply measuring the divergence between two distributions. However, it is difficult to interpret features using the max–sliced Wasserstein, where it is the key motivation of FROT.
Relation to Subspace Robust Wasserstein (Paty & Cuturi, 2019): Here, we show that 2-FRWD with d(x,y) = ‖x − y‖2 is a special case of SRW. Let us define U = ( √ α1e1, √ α2e2, . . . , √ αded)
> ∈ Rd×d, where e` ∈ Rd is the one-hot vector whose `th element is 1 and α>1 = 1, α` ≥ 0. Then, the objective function of SRW can be written as
n∑
i=1
m∑
j=1
πij‖U>xi −U>yj‖22 = n∑
i=1
m∑
j=1
πij(xi − yj)>UU>(xi − yj)
=
n∑
i=1
m∑
j=1
πij(xi − yj)>diag(α)(xi − yj)
=
n∑
i=1
m∑
j=1
πij
d∑
`=1
α`(x (`) i − y (`) j ) 2.
Therefore, SRW and 2-FRWD are equivalent if we set U = ( √ α1e1, √ α2e2, . . . , √ αded)
> and d(x,y) = ‖x− y‖2.

heading: ADDITIONAL SEMANTIC CORRESPONDENCE EXPERIMENTS
text: Figure 5a shows an example of key points matched using the FROT algorithm. Fig.5b shows the corresponding feature importance. The lower the η value, the smaller the number of layers used. The interesting finding here is that the selected important layer in this case is the third layer from the last. More qualitative results are presented in the Figure 7.

Review summaries:
This work proposes variants of robust OT/p-wasserstein-dist (3)/(4), where the ground cost is in some sense the maximum over costs with (prefixed) groups of features. The motivation is similar to that for feature selection: where perhaps only few of these groups of features are critical/sufficient for OT purposes. So it can also be understood as joint feature-group selection with OT. The resulting convex problem is proposed to be solved using FW, whose details are presented (including convergence).