SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Liangliang Shi, Haoyu Zhen, Gu Zhang, Junchi Yan*
Dept. of Computer Science and Engineering & MoE Key Lab of AI, Shanghai Jiao Tong University {shiliangliang, anye_zhen, blake-nash, yanjunchi}@sjtu.edu.cn PyTorch Code: https://github.com/LiangliangShi/RE-OT
Classification is a fundamental problem in machine learning, and considerable efforts have been recently devoted to the demanding long-tailed setting due to its prevalence in nature. Departure from the Bayesian framework, this paper rethinks classification from a matching perspective by studying the matching probability between samples and labels with optimal transport (OT) formulation. Specifically, we first propose a new variant of optimal transport, called Relative Entropic Optimal Transport (RE-OT), which guides the coupling solution to a known prior information matrix. We gives some theoretical results and their proof for RE-OT and surprisingly find RE-OT can help to deblur for barycenter images. Then we adopt inverse RE-OT for training long-tailed data and find that the loss derived from RE-OT has a similar form to Softmax-based cross-entropy loss, indicating a close connection between optimal transport and classification and the potential for transferring concepts between these two academic fields, such as barycentric projection in OT, which can map the labels back to the feature space. We further derive an epoch-varying RE-OT loss, and do the experiments on unbalanced image classification, molecule classification, instance segmentation and representation learning. Experimental results show its effectiveness.
Long-tailed label distribution or unbalanced classification in training given more balanced testing set has been a challenge for existing neural models [42, 3, 62]. Various approaches have been developed, such as Class Re-balancing [3, 25, 35, 8, 45, 52], Information Augmentation [36, 7], and Module Improvement [65, 16, 62]. However, the problem is still far from being solved.
Most existing methods on (long-tailed) classification study the conditional probability function  $p(y|x)$  or their joint distribution given the label  $y$  and sample  $x$ , almost taking it for granted. For instance, [45] shows that in the long-tailed case, according to Bayes' theorem, the vanilla Softmax classification is affected by the label distribution shift, which makes the classifier more inclined to consider the samples as belonging to the majority class. To address this issue, the work explicitly considers the label distribution shift and re-derives the Softmax function. [22] argues that the conditional distribution  $p(y|x)$  is not the same between training and testing data, which can result in target-shift. To enhance class-unbalanced learning, the authors propose to relax the assumption that the source and target domains share the same conditional distribution  $p(x|y)$ . Despite the de-facto Bayesian perspective for understanding classification, in this paper, we take a matching view and demonstrate how new insights and approaches can be derived, especially for the challenging long-tail cases.
*Correspondence author. This work was partly supported by NSFC (62222607, 61972250, U19B2035) and Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102).
From the matching perspective, the problem becomes matching samples to labels. During training, the goal is to learn the distances between the features of samples and labels. During testing, matching results can be obtained by calculating the distances between the features of testing samples and their labels. We formulate this matching using the theory of Optimal Transport (OT), where training precisely involves learning the Inverse OT [12, 33, 51], and testing involves optimizing the transportation results. In the commonly studied long-tailed setting [35], the labels of training samples follow a long-tail distribution, while the labels are uniformly distributed for testing data. With our OT formulation, we demonstrate that these two different priors (i.e. long-tailed and uniform) can effectively reduce the difference between training and testing, thereby increasing the accuracy of the matching probability.
To incorporate the prior information e.g. the aforementioned typical long-tailed setting, we introduce the smoothing-guided prior matrix  $\mathbf{Q}$  to vanilla entropic OT and call the resulting formulation relative entropic OT (RE-OT). To this new variant of OT, we provide convergence analysis, its corresponding static Schrödinger form, Sinkhorn algorithm for solving the problem as well as the dual formulation of RE-OT, which are mostly similar to those for the vanilla entropic OT. We find our RE-OT can help to deblur for barycenter images by setting specific smoothing prior  $\mathbf{Q}$ .
Specifically, we develop the inverse RE-OT (as our relative entropy OT's counterpart to inverse OT) to learn the cost matrix or equivalently, the sample features for training data. And then we use the RE-OT to optimize the transportation to get the matching results (i.e. predictions for testing data). It is uncovered that many Softmax-based cross-entropy losses (e.g. Balanced Softmax, A-softmax and etc) are special cases of our derived new loss under half constraints of OT by specifying different forms of the cost matrix, supervision matrix in IOT and the smoothing-guided prior matrix  $\mathbf{Q}$ . In other words, we can generalize the cross-entropy loss by specifying different cost matrix, smoothing-guided matrix, or other settings in the framework of inverse OT. We also find that when the regularization coefficient  $\epsilon \rightarrow 0$ , the loss of inverse RE-OT equals triplet loss in metric learning fields. Additionally, our matching perspective reveals interesting connections between OT and classification tasks. For example, we discover the equivalence between the coefficient of entropic regularization in OT and the temperature in Softmax. We also find that the concept of barycentric projection in OT can be used in classification to calculate the class barycenters in hidden feature space. We believe that the connection between these two fields will inspire further algorithmic development for both sides. This paper contributes in the following ways:
1) We propose a matching perspective to revisit and provide new understanding to the classification problem with OT formulation. The matching perspective provides a clear interpretation to the challenge of long-tailed classification, which lies in the inconsistency of the utilized prior information between the matches learned from the training set and those from the testing set.
2) We propose a new variant of entropic optimal transport, called relative entropic OT (RE-OT), for learning matching with a specified prior. We provide theoretical results on various aspects of RE-OT, including its solution properties, Sinkhorn algorithm, barycenter calculation, and its dual form. As a side product of our study, we find that RE-OT can help to deblur barycenter images by setting a specific smoothing prior  $\mathbf{Q}$ , which entropic OT alone cannot achieve.
3) We further develop Inverse RE-OT to link our results to classification, and show that Softmax and its variants are special cases of our derived loss based on Inverse RE-OT. This finding inspires a simple yet effective technique in practice for long-tailed classification: we let the smoothing-guided matrix  $\mathbf{Q}$  to be epoch-varying, which gradually provides prior information and improves the performance.
4) Our study has revealed a significant link between Optimal Transport (OT) and existing classification tasks, such as the relationship between the regularization coefficient in OT and the temperature in Softmax. This suggests that OT concepts can be applied to classification tasks, and vice versa. For instance, the barycentric projection concept in OT can be utilized in classification tasks to calculate the class barycenters in the hidden feature space. We anticipate that this connection between the two fields will lead to further algorithmic development for both OT and classification.
5) Our method performs competitively and often outperforms baselines on various long-tailed benchmarks ranging from image classification, to molecule property prediction, and instance segmentation. Source code will be made publicly available.
Optimal Transport. Known as Wasserstein distance or Earth Mover's distance, Optimal transport (OT) [26], has gained significant attention in various fields including domain adaption [11], generative models [15, 34, 49], and image registration [14]. We begin by briefly introducing discrete Optimal Transport (OT), which provides a framework for viewing the (long-tailed) classification problem from a matching perspective. For a more detailed introduction to OT, readers are directed to [43].
Consider two histograms  $\mathbf{a} \in \Sigma_n$  and  $\mathbf{b} \in \Sigma_m$ , where the simplex  $\Sigma_d = \{x \in \mathbb{R}_+^d | x^\top 1_d = 1\}$ . We can represent the transportation polytope  $U(\mathbf{a}, \mathbf{b})$ , which is the polyhedral set of  $n \times m$  matrices:
$$
U (\mathbf {a}, \mathbf {b}) = \left\{\mathbf {P} \in \mathbb {R} _ {+} ^ {n \times m} | \mathbf {P} \mathbf {1} _ {m} = \mathbf {a}, \mathbf {P} ^ {\top} \mathbf {1} _ {n} = \mathbf {b} \right\}, \tag {1}
$$
where  $\mathbf{1}_n$  and  $\mathbf{1}_m$  are  $n$  and  $m$  dimensional vectors of ones.  $U(\mathbf{a},\mathbf{b})$  contains all  $n\times m$  nonnegative probabilistic matrices with row and column sums a and b. Then the Kantorovich's optimal transport problem can be defined as [26]:
$$
\min  _ {\mathbf {P} \in U (\mathbf {a}, \mathbf {b})} <   \mathbf {C}, \mathbf {P} > = \sum_ {i, j} \mathbf {C} _ {i j} \mathbf {P} _ {i j}, \tag {2}
$$
where  $\mathbf{C} \in \mathbb{R}_+^{n \times m}$  is the cost matrix for the distance between sample  $\{\mathbf{x}_i\}_{i=1}^n$  and  $\{\mathbf{y}_j\}_{j=1}^m$ . The above minimization is a linear program and the optimal solution  $\mathbf{P}^*$  can be obtained with the network simplex [54] or other off-the-shelf techniques e.g. [40], which require significant time overhead. For its cost-effectiveness, the entropic regularization [57] is more popular and it gets an approximation:
$$
\min  _ {\mathbf {P} \in U (\mathbf {a}, \mathbf {b})} \mathcal {L} ^ {\epsilon} = <   \mathbf {C}, \mathbf {P} > - \epsilon H (\mathbf {P}), \tag {3}
$$
where the regularizer is specified as  $H(\mathbf{P}) = \sum_{ij} -\mathbf{P}_{ij}(\log (\mathbf{P}_{ij}) - 1)$ . It can be proved that [43] when  $\epsilon \rightarrow 0$ , the unique solution of Eq. 3 converges to the optimal  $\mathbf{P}^*$  of original problem. And when  $\epsilon \rightarrow +\infty$ , we can get
$$
\mathbf {P} _ {\epsilon} \stackrel {\epsilon \rightarrow + \infty} {\longrightarrow} \mathbf {a} \otimes \mathbf {b}, \tag {4}
$$
where  $\mathbf{a} \otimes \mathbf{b} = \mathbf{a}^{\top}\mathbf{b} \in U(\mathbf{a},\mathbf{b})$ . So we can find the entropic regularization  $H(\mathbf{P})$  exactly as a smoothing to the solution towards  $\mathbf{a} \otimes \mathbf{b}$ . Recently, Inverse Optimal Transport (IOT) starts to be studied [12, 33, 51, 50] which assumes that the cost  $\mathbf{C}$  is unknown for learning, given the established coupling. The inverse of entropic OT problem can be formulated as:
$$
\min  _ {\theta} K L (\tilde {\mathbf {P}} | \mathbf {P} ^ {\theta}), \quad \text {w h e r e} \quad \mathbf {P} ^ {\theta} = \arg \min  _ {\mathbf {P} \in U (\mathbf {a}, \mathbf {b})} <   \mathbf {C} ^ {\theta}, \mathbf {P} > - \epsilon H (\mathbf {P}). \tag {5}
$$
Specifically, [51] proposes a principled approach to infer the unknown costs and [6] develops the mathematical theory for IOT. [33] shows that by IOT, one can also modify and improve the raw matching prediction by the cost, which is learned by the raw matching as noise supervision.
In this paper, we develop an IOT-based formulation to learn the cost between samples and their labels, which further allows one to learn the feature extractor of the training samples. Then with the features exacted from testing data, we match them with target labels and achieve the goal of classification.
Long-tailed recognition. Long-tailed recognition has been a standing problem with wide attention in vision and learning [64, 19, 35]. One common approach [45, 41] is to re-balance the class distribution in the training set. For example, some works employ re-sampling [25] or re-weighting [8] techniques to let the model pay more attention to the minority classes in the training set. Data augmentation is also adopted to synthesize more samples for the underrepresented classes [28]. While in [9], transfer learning is used, which leverages knowledge learned from a source domain with sufficient data to improve the performance on a target domain with imbalanced data. This line of research includes techniques such as fine-tuning [63], domain adaptation [53], and meta-learning [56]. Recently, some works have proposed to explicitly model the long-tailed distribution, such as by adjusting the loss or designing specialized network architectures. Examples include focal loss [35], class-balanced loss [8], balanced Softmax loss [45]. In this paper, we view classification from a matching perspective. Our method does not predict the label or class directly but learns the features in training and calculates the matching probability instead. With the aid of OT, we formulate it with relative entropic as proposed in the next section, which adopts different prior for training and testing.
$\epsilon = 0.1$
Figure 1: Results of  $\mathbf{P}_{\mathbf{Q}}^{\epsilon}$  by varying  $\epsilon$  given the features (trained with SimCLR [5]) of ten CIFAR-10 images and their augmentation. The cost matrix is set with the cosine distance between features. When  $\epsilon \rightarrow 0$ ,  $\mathbf{P}_{\mathbf{Q}}^{\epsilon}$  becomes sharper for probability prediction and when  $\epsilon \rightarrow +\infty$ ,  $\mathbf{P}_{\mathbf{Q}}^{\epsilon}$  is more close to  $\tilde{\mathbf{Q}}$ . Here we set  $\mathbf{Q}_{ij} = 0.7$  if sample  $i$  and  $j$  are in the same class and  $\mathbf{Q}_{ij} = 0.3$  otherwise.
$\epsilon = 1$
$\epsilon = 10$
3 Relative Entropy Regularization for Optimal Transport
Motivation of introducing prior into OT. As introduced in Eq. 3, entropic regularization (ER) is an effective method [10] for approximating the Kantorovich OT solution. It pushes the original linear programming solution away from the hard boundary and obtains a smoother solution by minimizing a  $\epsilon$ -strongly convex function. With the increase of the penalty coefficient, the solution progressively approaches a more 'smooth' solution, ultimately converging to  $\mathbf{a}^{\top}\mathbf{b}$  as illustrated by Eq. 4. However, a question is raised: is such a smoothing direction  $\mathbf{a}^{\top}\mathbf{b}$  appropriate for all practical situations, e.g., the long-tailed problems? Can we freely choose a more suitable (smoothing) direction to achieve more effective results tailored to the problem or dataset at hand? In this section, we will develop relative entropic OT (RE-OT), a more general formulation than traditional entropic OT [57].
**Formulation.** Given a cost matrix  $\mathbf{C} \in \mathbb{R}_{+}^{n \times m}$  and two margins  $\mathbf{a} \in \Sigma_{n}$  and  $\mathbf{b} \in \Sigma_{m}$ , where  $\Sigma_{n}$  and  $\Sigma_{m}$  represent the probability simplex with  $n$  and  $m$  bins, respectively, we introduce the positive smoothing-guidance matrix  $\mathbf{Q}$  as a manually-specified constant prior to guide the smoothing. We use the relative entropy regularizer  $H_{\mathbf{Q}}(\mathbf{P})$  in place of the traditional entropy in Eq. 3 to achieve this ability, defined as  $H_{\mathbf{Q}}(\mathbf{P}) = -\sum_{ij} \mathbf{P}_{ij} \left( \log \frac{\mathbf{P}_{ij}}{\mathbf{Q}_{ij}} - 1 \right)$ . Then we have:
$$
\min  _ {\mathbf {P} \in U (\mathbf {a}, \mathbf {b})} \mathcal {L} _ {\mathbf {Q}} ^ {\epsilon} = <   \mathbf {C}, \mathbf {P} > - \epsilon H _ {\mathbf {Q}} (\mathbf {P}). \tag {6}
$$
The relative entropy  $H_{\mathbf{Q}}(\mathbf{P})$  can increase the probability of transportation for the element  $\mathbf{P}_{ij}$  if  $\mathbf{Q}_{ij}$  is larger. Consequently, if the prior in the form of  $\mathbf{Q}$  is available, the use of relative entropy regularization can improve the efficiency of learning the transportation solution. Therefore, for long-tailed tasks, we can assign a large  $\mathbf{Q}$  element to the majority classes and a low element of  $\mathbf{Q}$  to the tailed classes to enhance training. We will discuss this in detail in the next section.
For Eq. 6, we further study its properties. Firstly, we show the relation between RE-OT and the entropic OT by Eq. 3 as follows. The proof is given in Table A.
Proposition 1 (Solution Propriety). Assume  $\mathbf{P}_Q^\epsilon$ ,  $\mathbf{P}^{\epsilon}$  and are the optimal solution of RE-OT and entropic regularized OT, respectively. Then we can get:
(1) When  $\epsilon \rightarrow +\infty$ , the optimal RE-OT's solution  $\mathbf{P}_Q^\epsilon$  will converge to  $\widetilde{\mathbf{Q}}$  where  $\widetilde{\mathbf{Q}}$  takes the form  $\widetilde{\mathbf{Q}} = \text{diag}(\mathbf{u})\mathbf{Q}\text{diag}(\mathbf{v})$  with two uniquely defined non-negative vectors  $\mathbf{u}$  and  $\mathbf{v}$ .
(2) With the prior  $\mathbf{Q}$  and its corresponding  $\widetilde{\mathbf{Q}}$  as defined in (1), we have  $\mathbf{P}_{\mathbf{Q}}^{\epsilon} = \mathbf{P}_{\widetilde{\mathbf{Q}}}^{\epsilon}$ . And when  $\widetilde{\mathbf{Q}} = \mathbf{a} \otimes \mathbf{b}$ , we have the equality  $\mathbf{P}^{\epsilon} = \mathbf{P}_{\widetilde{\mathbf{Q}}}^{\epsilon}$ .
Proposition 1 shows the relation between RE-OT and entropic OT, and we can find RE-OT is a more general formulation. Note  $\widetilde{\mathbf{Q}}$  is defined by  $\mathbf{Q}$  and can be obtained by iteratively normalizing row/column sums of  $\mathbf{Q}$ . Fig. 1 shows the results of  $\mathbf{P}_{\mathbf{Q}}^{\epsilon}$  by varying  $\epsilon$ . As  $\epsilon$  increases, the optimal coupling becomes more dense, but  $\mathbf{P}^{\epsilon}$  becomes more uniform while  $\mathbf{P}_{\mathbf{Q}}^{\epsilon}$  converges to  $\widetilde{\mathbf{Q}}$ .
Besides, similar to the standard entropic regularized OT, our RE-OT in Eq. 6 can also be reformulated to the "static Schrödinger problem" form [31], which exactly leans a projection under KL divergence.
Proposition 2 (static Schrödinger form). Redefine a general KL divergence as
$$
\widetilde {K L} (\mathbf {P} | \mathbf {K}) = \sum_ {i j} \mathbf {P} _ {i j} \log \frac {\mathbf {P} _ {i j}}{\mathbf {K} _ {i j}} - \mathbf {P} _ {i j} + \mathbf {K} _ {i j}, \tag {7}
$$
The optimization in Eq. 6 is equivalent to the following minimization, where  $\mathbf{K}_{ij} = \mathbf{Q}_{ij}e^{-\mathbf{C}_{ij} / \epsilon}$
$$
\mathbf {P} _ {Q} ^ {\epsilon} = \underset {\mathbf {P} \in U (\mathbf {a}, \mathbf {b})} {\arg \min } \widetilde {K L} (\mathbf {P} | \mathbf {K}). \tag {8}
$$
The proof is in Appendix B. Prop. 2 shows the optimal solution  $\mathbf{P}_Q^\epsilon$  is exactly the KL projection of  $\mathbf{K}$  onto  $U(\mathbf{a},\mathbf{b})$ . In fact,  $\mathbf{K}$  can be set as  $\mathbf{K}_{ij} = \widetilde{\mathbf{Q}}_{ij}e^{-\mathbf{C}_{ij} / \epsilon}$  due to the equivalence between  $\mathbf{P}_Q^\epsilon$  and  $\mathbf{P}_{\widetilde{\mathbf{Q}}}^\epsilon$ . Expect for the different form of  $\mathbf{K}$ , RE-OT and entropic OT share the same static Schrödinger formulation, which means that many methods in entropic OT can be applied to RE-OT, such as iterative Bregman projections [2].
The Sinkhorn algorithm and barycenters for RE-OT. The optimal solution for RE-OT can be estimated using the Sinkhorn algorithm, as its counterpart has already been well-developed for solving entropic OT [10]. The algorithm for RE-OT shares the same form as that for entropic OT, given  $\mathbf{K}_{ij} = \widetilde{\mathbf{Q}}_{ij}e^{-\mathbf{C}_{ij} / \epsilon}$ . Specifically, with two non-negative vectors  $\mathbf{u}'$  and  $\mathbf{v}'$  uniquely defined up to a multiplicative factor, the optimal solution has the form  $\mathbf{P}_Q^\epsilon = \mathrm{diag}(\mathbf{u}')\mathbf{K}\mathrm{diag}(\mathbf{v}')$ , which can be efficiently computed by iterating  $\mathbf{u}'$ ,  $\mathbf{v}' \gets \mathbf{a}.\mathbf{/K}\mathbf{v}'$ ,  $\mathbf{b}.\mathbf{/K}^\top \mathbf{u}'$ . The proof and algorithm are presented in Appendix C. In addition, the barycenter between distributions is a natural extension of OT [1, 2], and with the matrix  $\mathbf{K}$ , [2] defined it with a weighted KL projection problem:
$$
\min _ {\left\{\mathbf {P} _ {s} \right\}, \mathbf {a}} \sum_ {s} \epsilon \cdot \lambda_ {s} \widetilde {K L} (\mathbf {P} _ {s} | \mathbf {K}) \text {s . t .} \mathbf {P} _ {s} ^ {\top} \mathbf {1} _ {n} = \mathbf {b} _ {s}, \mathbf {P} _ {1} \mathbf {1} _ {m} = \mathbf {P} _ {2} \mathbf {1} _ {m} = \dots = \mathbf {P} _ {S} \mathbf {1} _ {m} = \mathbf {a}, \tag {9}
$$
where  $\{\lambda_s\}_{s=1}^S$  are the weights,  $\mathbf{b}_s$  are known histograms representing images, and  $\mathbf{a}$  is the barycenter histogram to be calculated. Though easier to calculate compared with the regularized formulation in [1], a drawback of this entropic-based method is that it can lead to blurred barycenters, and methods have been proposed to address this issue [23]. By calculating barycenters between noise and an image (i.e.  $S = 2$  here), we show that the blurred problem can be simply solved by setting  $\lambda$  where  $\lambda_1 = \lambda, \lambda_2 = 1 - \lambda$  and the prior matrix  $\mathbf{Q}$ :
$$
\mathbf {Q} = (1 - \lambda) \left(\mathbf {P} ^ {\epsilon}\right) ^ {\top} + \lambda \mathbf {P} ^ {\epsilon}, \tag {10}
$$
where  $\mathbf{P}^{\epsilon}$  is the optimal solution of entropic OT from image  $\mathbf{b}_1$  to  $\mathbf{b}_2$ . Fig. 2 illustrates the effect of different choices of  $\mathbf{Q}$  when computing barycenters between noise and an image. Without using  $\mathbf{Q}$ , the resulting barycenters are very blurry. When we set  $\mathbf{Q}$  to be  $\mathbf{P}^{\epsilon}$ , the barycenters display the shape of the leopard clearly, but fail to transition smoothly from noise to leopard image as  $\lambda$  changes. To address this issue, we set  $\mathbf{Q}$  as shown in Eq. 10, which can blur the image while transitioning smoothly with the gradient.
Proposition 3 (Dual formulation). From the optimization in Eq. 6, we can get its dual formulation:
$$
L _ {\mathbf {Q}} ^ {\epsilon} = <   \mathbf {f}, \mathbf {a} > + <   \mathbf {g}, \mathbf {b} > - \epsilon <   e ^ {\mathbf {f} / \epsilon}, \mathbf {K} e ^ {\mathbf {g} / \epsilon} > \tag {11}
$$
where  $\mathbf{f} \in \mathbb{R}^n$  and  $\mathbf{g} \in \mathbb{R}^m$  are the corresponding dual variables.
The proof is given in Appendix D. Exactly  $\mathbf{f}$ ,  $\mathbf{g}$  are linked to  $\mathbf{u}$ ,  $\mathbf{v}$  appearing in Sinkhorn algorithm by  $\mathbf{u} = \exp(\mathbf{f}/\epsilon)$  and  $\mathbf{v} = \exp(\mathbf{g}/\epsilon)$ , and thus Sinkhorn algorithm can be done in log-domain [47].
Setting  $\mathbf{Q}$  with the Optimal Solution Iteratively. The intuition for the prior is to iteratively update the solution  $P_{\mathbf{Q}}^{\epsilon}$  as a new  $\mathbf{Q}$ , i.e.,  $\mathbf{Q}^{(t)} = \mathbf{P}_{\mathbf{Q}^{(t - 1)}}^{\epsilon}$ , where  $\mathbf{Q}^{(t)}$  represents the solution obtained after the  $n$ -th iteration. The question then arises as to how  $\mathbf{Q}^{(t)}$  changes over time. We find that this problem is equivalent to a proximal point algorithm:
$$
\mathbf {Q} ^ {(t)} = \underset {\mathbf {Q}} {\arg \min } K L (\mathbf {Q} | \mathbf {Q} ^ {(t - 1)}) + \frac {1}{\epsilon} F (\mathbf {Q}). \tag {12}
$$
Here  $F(\mathbf{Q}) = \langle C, \mathbf{Q} \rangle + l_{U(\mathbf{a},\mathbf{b})}(\mathbf{Q})$ , where  $l_{U(\mathbf{a},\mathbf{b})}(\mathbf{Q})$  is defined such that  $l_{U(\mathbf{a},\mathbf{b})}(\mathbf{Q}) = 0$  if  $\mathbf{Q} \in U(\mathbf{a},\mathbf{b})$ , and  $l_{U(\mathbf{a},\mathbf{b})}(\mathbf{Q}) = +\infty$  otherwise. We find this problem is discussed by [29, 43],