SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Chudi Zhong $^{1*}$  Zhi Chen $^{1*}$  Jiachang Liu $^{1}$  Margo Seltzer $^{2}$  Cynthia Rudin $^{1}$
<sup>1</sup> Duke University <sup>2</sup> The University of British Columbia {chudi.zhong, zhi.chen1, jiachang.liu}@duke.edu.msaltzer@cs.ubc.ca, cynthiaa@cs.duke.edu
In real applications, interaction between machine learning models and domain experts is critical; however, the classical machine learning paradigm that usually produces only a single model does not facilitate such interaction. Approximating and exploring the Rashomon set, i.e., the set of all near-optimal models, addresses this practical challenge by providing the user with a searchable space containing a diverse set of models from which domain experts can choose. We present algorithms to efficiently and accurately approximate the Rashomon set of sparse, generalized additive models with ellipsoids for fixed support sets and use these ellipsoids to approximate Rashomon sets for many different support sets. The approximated Rashomon set serves as a cornerstone to solve practical challenges such as (1) studying the variable importance for the model class; (2) finding models under user-specified constraints (monotonicity, direct editing); and (3) investigating sudden changes in the shape functions. Experiments demonstrate the fidelity of the approximated Rashomon set and its effectiveness in solving practical challenges.
A key ingredient for trust in machine learning models is interpretability; it is more difficult to trust a model whose computations we do not understand. However, building interpretable models is not easy; even if one creates a model that is sparse and monotonic in the right variables, it likely is still not what a domain expert is looking for. In fact, domain experts often cannot fully articulate the constraints their problem requires. This leads to an often painful iterative process where a machine learning algorithm, which is typically designed to produce only one model, is now asked to produce another, and possibly many more after that. Hence, the classical machine learning paradigm is broken, because it was not designed to facilitate choice among models. In other words, the classical machine learning paradigm was not designed for the real-world situation in which domain experts see and understand a model, identify shortcomings in it, and change it interactively in a specific way. There is instead an interaction bottleneck.
We work in a new paradigm where an algorithm produces many models from which to choose, instead of just one [1-3]. Algorithms in this new paradigm produce or approximate the Rashomon set of a given function class. The Rashomon set is the set of models that are approximately as good as the best model in the class. That is, it is the set of all good models. Rashomon sets for many real problems are surprisingly large [4-6], and there are theoretical reasons why we expect many simple, good models to exist [3]. The question we ask here is how to explicitly find the Rashomon set for the class of sparse generalized additive models (sparse GAMs).
GAMs are one of the most widely used forms of interpretable predictive models [7-9] and have been applied to complex problems such as prediction of health outcomes from medical records [10, 11], where they can provide the same accuracy as the best black box models. GAMs can use sparsity regularization to generalize and to avoid constructing overly complicated models. Different regularization strategies have been used [8, 12, 13], and sparse GAMs have performance that is essentially identical to black box models on most tabular datasets [12]. Thus, an important previously-missing piece of the Rashomon set paradigm is how to obtain the Rashomon set of sparse GAMs.
There are two major challenges: (1) Different from regression, the Rashomon set of GAMs in the classification setting has no analytical form; (2) The Rashomon set should lend itself easily to human-model interaction, mitigating the interaction bottleneck. To tackle these challenges, we find the maximum volume ellipsoid inscribed in the true Rashomon set. Our gradient-based optimization algorithm can find such an ellipsoid for a fixed support set. Leveraging the properties of ellipsoids, we can efficiently approximate Rashomon sets of numerous support sets that are subsets of the original support set. We show in Section 5 that this approximation captures most of the Rashomon set while including few models outside of it.
More importantly, using the maximum volume inscribed ellipsoid enables us to solve many practical challenges of GAMs through efficient convex optimization. We first use it to study the importance of variables among a set of well-performing models, called variable importance range. This is essentially the Model Class Reliance, which measures the variable importance of models in the Rashomon set [14, 15]; previously, ranges of variable importance have been studied only for linear models [14] and tree models [2, 16]. We also show how to efficiently use the ellipsoid approximation to search for near-optimal models with monotonicity constraints. The Rashomon set empowers users to be able to arbitrarily manipulate models. As users edit models, their edits are either already in the Rashomon set (which we can check easily) or they can easily be projected back into the Rashomon set using the technique in Section 4.4. Thus, being able to approximate the full Rashomon set for GAMs brings users entirely new functionality, going way beyond what one can do with just a set of diverse models [17-23]. Also, the ability to sample GAMs efficiently from the approximated Rashomon set allows us to investigate whether the variations or sudden changes observed in a single model are indicative of true patterns in the dataset or simply random fluctuations.
Since our work provides the first method for constructing Rashomon sets for sparse GAMs, there is no direct prior work; this is a novel problem. There has been prior work on related problems, such as constructing Rashomon sets for decision trees [2], and the Rashomon sets for linear regression are simply described by ellipsoids [3]; Rashomon sets for linear regression models have been used for decision making [24], robustness of estimation [25], and holistic understanding of variable importance [14, 26]. Our work allows these types of studies to generalize to GAMs.
A sample is denoted as  $(\mathbf{x},y)$ , where  $\mathbf{x}$  is the  $p$  dimensional feature vector and  $y$  is the target. The  $j^{th}$  dimension of the feature vector is  $x_{j}$ . A generalized additive model (GAM) [7] has the form
$$
g (E [ y ]) = \omega_ {0} + f _ {1} \left(x _ {1}\right) + f _ {2} \left(x _ {2}\right) + \dots + f _ {p} \left(x _ {p}\right) \tag {1}
$$
where  $\omega_0$  is the intercept,  $f_{j}$ 's are the shape functions and  $g$  is the link function, e.g., the identity function for regression or the logistic function for classification. Each shape function  $f_{j}$  operates only on one feature  $x_{j}$ , and thus the shape function can directly be plotted. This makes GAMs interpretable since the entire model can be visualized via 2D graphs. In practice, a continuous feature is usually divided into bins [8], thereby its shape function can be viewed as a step function, i.e.,
$$
f _ {j} \left(x _ {j}\right) = \sum_ {k = 0} ^ {B _ {j} - 1} \omega_ {j, k} \cdot \mathbf {1} \left[ b _ {j, k} <   x _ {j} \leq b _ {j, k + 1} \right], \tag {2}
$$
where  $\{b_{j,k}\}_{k = 0}^{B_j}$  are the bin edges of feature  $j$ , leading to  $B_{j}$  total bins. This is a linear function on the binned dataset whose features are one-hot vectors.  $\omega = \{\omega_{j,0},\omega_{j,1},\dots ,\omega_{j,B_j - 1}\}_{j = 1}^p$  is the weight vector. For simplicity, we use this formulation in the rest of this paper.
Given a dataset  $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^n$ , we use the logistic loss  $\mathcal{L}_c(\omega, \omega_0, \mathcal{D})$  as the classification loss. To regularize the shape function, we also consider a weighted  $\ell_2$  loss  $\mathcal{L}_2(\omega)$  on the coefficients and
total number of steps in the shape functions  $\mathcal{L}_s(\omega)$  as penalties. Specifically,
$$
\mathcal {L} _ {2} (\omega) := \sum_ {j = 1} ^ {p} \sum_ {k = 0} ^ {B _ {j} - 1} \pi_ {j, k} \omega_ {j, k} ^ {2},
$$
where  $\pi_{j,k} = \frac{1}{n}\sum_{i = 1}^{n}\mathbf{1}[b_{j,k} < x_{ij}\leq b_{j,k + 1}]$  is the proportion of samples in bin  $k$  of feature  $j$ . The weighted  $\ell_2$  term not only penalizes the magnitude of the shape function, but also implicitly centers the shape function by setting the population mean to 0: any other offsets would lead to suboptimal  $\mathcal{L}_2(\omega)$ . This approach is inspired by [8], which explicitly sets the population mean to 0. To make the shape functions fluctuate less, we penalize the total number of steps in all shape functions, i.e.,
$$
\mathcal {L} _ {s} (\boldsymbol {\omega}) = \sum_ {j = 1} ^ {p} \sum_ {k = 0} ^ {B _ {j} - 1} \mathbf {1} \left[ w _ {j, k} \neq w _ {j, k + 1} \right],
$$
which is similar to the  $\ell_0$  penalty of [12]. Combining classification loss and penalty yields total loss:
$$
\mathcal {L} (\omega , \omega_ {0}, \mathcal {D}) = \mathcal {L} _ {c} (\omega , \omega_ {0}, \mathcal {D}) + \lambda_ {2} \mathcal {L} _ {2} (\omega) + \lambda_ {s} \mathcal {L} _ {s} (\omega). \tag {3}
$$
Following the definition of [3], we define the Rashomon set of sparse GAMs as follows:
Definition 2.1. ( $\theta$ -Rashomon set) For a binned dataset  $\mathcal{D}$  with  $n$  samples and  $m$  binary features, where  $\omega \in \mathbb{R}^m$  defines a generalized additive model. The  $\theta$ -Rashomon set is a set of all  $\omega \in \mathbb{R}^m$  with  $\mathcal{L}(\omega, \omega_0, \mathcal{D})$  at most  $\theta$ :
$$
R (\theta , \mathcal {D}) := \left\{\omega \in \mathbb {R} ^ {m}, \omega_ {0} \in \mathbb {R}: \mathcal {L} \left(\omega , \omega_ {0}, \mathcal {D}\right) \leq \theta \right\}. \tag {4}
$$
We use  $\mathcal{L}(\omega, \omega_0)$  to represent  $\mathcal{L}(\omega, \omega_0, \mathcal{D})$  and  $R(\theta)$  to represent  $R(\theta, \mathcal{D})$  when  $\mathcal{D}$  is clearly defined.
3 Approximating the Rashomon Set
3.1 GAM with Fixed Support Set (Method 1)
Suppose we merge all adjacent bins with the same  $\omega_{j}$ . The set of bins after merging is called the support set. For a GAM with fixed support set, the  $\mathcal{L}_s$  term is fixed and the problem is equivalent to logistic regression with a weighted  $\ell_2$  penalty. For simplicity, in what follows,  $\omega$  also includes  $\omega_0$ . In this case, the loss  $\mathcal{L}(\omega)$  is a convex function, and the Rashomon set  $R(\theta)$  is a convex set. Hence, we find the maximum volume inscribed ellipsoid centered at  $\omega_c$  to approximate  $R(\theta)$ . Specifically, the approximated Rashomon set is:
$$
\hat {R} := \left\{\boldsymbol {\omega} \in \mathbb {R} ^ {m}: \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) ^ {T} \mathbf {Q} \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) \leq 1 \right\}, \tag {5}
$$
where  $\mathbf{Q}$  and  $\omega_{c}$  are the parameters defining the ellipsoid that we optimize given different  $\theta$ s.
Gradient-based optimization: Our goal is to maximize the volume of  $\hat{R}$  while guaranteeing that almost all points in  $\hat{R}$  are also in  $R(\theta)$ . Mathematically, we define our optimization objective as
$$
\min  _ {\mathbf {Q}, \boldsymbol {\omega} _ {c}} \det  (\mathbf {Q}) ^ {\frac {1}{2 m}} + C \cdot \mathbb {E} _ {\boldsymbol {\omega} \sim \hat {R} (\mathbf {Q}, \boldsymbol {\omega} _ {c})} [ \max  (\mathcal {L} (\boldsymbol {\omega}) - \theta , 0) ]. \tag {6}
$$
The volume of  $\hat{R}$  is proportional to  $\operatorname*{det}(\mathbf{Q})^{-1/2}$ , therefore we minimize  $\operatorname*{det}(\mathbf{Q})^{1/(2m)}$  in the objective.  $m$  is included in the exponent to normalize by the number of dimensions. In the second term of Eq (6), we draw samples from  $\hat{R}(\mathbf{Q}, \omega_c)$  and add a penalty if the sample is outside  $R(\theta)$ ;  $C$  is large to ensure this happens rarely. We use gradient descent to optimize the proposed objective function. Note that the "reparameterization trick" can be used to convert the sample  $\omega$  drawn from  $\hat{R}$  into a purely stochastic part and a deterministic part related to  $\mathbf{Q}$  and  $\omega_c$ , and calculate the gradients accordingly. Details are shown in Appendix B. To facilitate the optimization, we need a good starting point.
Initialization: Since the support set is fixed, we treat  $\mathcal{L}_s$  as a constant. We initialize  $\omega_{c}$  using the empirical risk minimizer  $\omega^{*}$  for  $\mathcal{L}(\omega)$  and initialize  $\mathbf{Q}$  by  $\mathbf{H} / (2(\theta -\mathcal{L}(\omega^{*})))$  , where  $\mathbf{H}$  is the Hessian. The initialization is motivated by the Taylor expansion on  $\mathcal{L}(\omega)$  at the empirical risk minimizer  $\omega^{*}$  ..
$$
\mathcal {L} (\boldsymbol {\omega}) \approx \mathcal {L} (\boldsymbol {\omega} ^ {*}) + \nabla \mathcal {L} ^ {T} (\boldsymbol {\omega} - \boldsymbol {\omega} ^ {*}) + \frac {1}{2} (\boldsymbol {\omega} - \boldsymbol {\omega} ^ {*}) ^ {T} \mathbf {H} (\boldsymbol {\omega} - \boldsymbol {\omega} ^ {*}) + O ((\boldsymbol {\omega} - \boldsymbol {\omega} ^ {*}) ^ {3}).
$$
Since  $\omega^{*}$  is the empirical risk minimizer, the gradient  $\nabla \mathcal{L} = 0$ . If we ignore the higher order terms, we get a quadratic function, i.e.,  $\mathcal{L}(\omega)\approx \mathcal{L}(\boldsymbol{\omega}^{*}) + \frac{1}{2} (\boldsymbol {\omega} - \boldsymbol{\omega}^{*})^{T}\mathbf{H}(\boldsymbol {\omega} - \boldsymbol{\omega}^{*})$ . Plugging this quadratic function into the Rashomon set formula, we get exactly an ellipsoid, i.e.,
$$
\frac {1}{2} (\boldsymbol {\omega} - \boldsymbol {\omega} ^ {*}) ^ {T} \mathbf {H} (\boldsymbol {\omega} - \boldsymbol {\omega} ^ {*}) \leq \theta - \mathcal {L} (\boldsymbol {\omega} ^ {*}).
$$
Therefore, given this approximation, we initialize  $\omega_{c}$  with  $\omega^{*}$  and  $\mathbf{Q}$  with  $\mathbf{H} / (2(\theta -\mathcal{L}(\boldsymbol{\omega}^{*})))$  , and then optimize through gradient descent.
3.2 GAMs with Different Support Sets (Method 2)
Method 1 can accurately approximate the Rashomon set given a fixed support set. However, if we want the Rashomon set for many different support sets, applying Method 1 repeatedly can be time-consuming. We next present a blocking method, Method 2, to efficiently approximate the Rashomon sets with many smaller support sets after getting an approximated Rashomon set for a large support set by Method 1. Approximating the Rashomon set with an ellipsoid for a large support can be helpful in approximating the Rashomon sets with smaller support sets. Specifically, if the bins of the smaller support set are generated by merging bins in the original support set, it is equivalent to forcing the coefficients of these merged bins to be the same. Formally, this is a hyperplane  $P$  in the solution space defined by a set of linear constraints
$$
P := \left\{\omega \in \mathbb {R} ^ {m}: \omega_ {k} = \omega_ {k + 1} = \dots = \omega_ {\kappa} \right\}, \tag {7}
$$
where the  $k^{th}$  to the  $\kappa^{th}$  bins in the original support set are merged to create a single bin in the new support set. The hyperplane's intersection with any ellipsoid, e.g.,  $\hat{R}$ , is still an ellipsoid, which can be calculated analytically. Specifically, if we rewrite  $\mathbf{Q}$  in the form of block matrices, i.e.,
$$
\mathbf {Q} = \left[ \begin{array}{c c c} \mathbf {Q} _ {1: k - 1, 1: k - 1} & \mathbf {Q} _ {1: k - 1, k: \kappa} & \mathbf {Q} _ {1: k - 1, \kappa + 1: m} \\ \hline \mathbf {Q} _ {1: k - 1, k: \kappa} ^ {T} & \mathbf {Q} _ {k: \kappa , k: \kappa} & \mathbf {Q} _ {\kappa + 1: m, k: \kappa} ^ {T} \\ \hline \mathbf {Q} _ {1: k - 1, \kappa + 1: m} ^ {T} & \mathbf {Q} _ {\kappa + 1: m, k: \kappa} & \mathbf {Q} _ {\kappa + 1: m, \kappa + 1: m} \end{array} \right],
$$
we obtain the quadratic matrix  $\tilde{\mathbf{Q}}$  for the new support set by summing up the  $k^{th}$  to  $\kappa^{th}$  rows and columns, i.e.,
$$
\tilde {\mathbf {Q}} = \left[ \begin{array}{c c c} \mathbf {Q} _ {1: k - 1, 1: k - 1} & \mathbf {q} _ {1: k - 1} & \mathbf {Q} _ {1: k - 1, \kappa + 1: m} \\ \hline \mathbf {q} _ {1: k - 1} ^ {T} & q _ {\mathrm {m i d}} & \mathbf {q} _ {k + 1: m} ^ {T} \\ \hline \mathbf {Q} _ {1: k - 1, \kappa + 1: m} ^ {T} & \mathbf {q} _ {k + 1: m} & \mathbf {Q} _ {\kappa + 1: m, \kappa + 1: m} \end{array} \right],
$$
where  $q_{\mathrm{mid}} = \sum_{i=k}^{\kappa} \sum_{j=k}^{\kappa} Q_{i,j}$ ,  $\mathbf{q}_{1:k-1:m}$  and  $\mathbf{q}_{\kappa+1:m}$  are two column vectors obtained by summing all columns of  $\mathbf{Q}_{1:k-1,k:\kappa}$  and  $\mathbf{Q}_{\kappa+1:m,k:\kappa}$ . The linear term (coefficients on the  $\omega$  term) of the original support set is  $\ell := \mathbf{Q}\omega_c$ . Similarly, we can obtain the linear term of the new support set  $\tilde{\ell}$  by summing up the  $k^{th}$  to  $\kappa^{th}$  elements of the  $\ell$  vector, i.e.,  $\tilde{\ell} = [\ell_1, \dots, \ell_{k-1}, \sum_{i=k}^{\kappa} \ell_i, \ell_{\kappa+1}, \dots, \ell_m]^T$ . With this, we can calculate the new ellipsoid's center,  $\tilde{\omega}_c = \tilde{\mathbf{Q}}^{-1}\tilde{\ell}$ , and the upper bound of the new ellipsoid  $u = 1 - \omega_c^T\mathbf{Q}\omega_c + \tilde{\omega}_c^T\tilde{\mathbf{Q}}\tilde{\omega}_c$ . This new ellipsoid  $R_P := P \cap \hat{R}$ , which satisfies both Equations 5 and 7, is defined as
$$
R _ {P} := \{\pmb {\omega} \in \mathbb {R} ^ {m}: (\pmb {\omega} - \tilde {\pmb {\omega}} _ {c}) ^ {T} \tilde {\mathbf {Q}} (\pmb {\omega} - \tilde {\pmb {\omega}} _ {c}) \leq u \},
$$
where if  $u \leq 0$ ,  $R_P$  is empty. If we want to merge multiple sets of bins, the calculations can be slightly modified to summing up sets of rows and columns separately corresponding to the sets of bins to be merged.  $R_P$  serves as a simple approximation of the Rashomon set for the smaller support set. Since essentially all of the original ellipsoid  $\hat{R}$  is in the Rashomon set, and  $R_P$  is a subset of  $\hat{R}$ , every  $\omega$  in  $R_P$  is also in the Rashomon set. In fact, since the support set gets smaller,  $\mathcal{L}_s$  becomes smaller too, and the loss is even lower, i.e., for  $\omega$  in  $R_P$ ,  $\mathcal{L}(\omega) \leq \theta - \lambda_0(\kappa - k)$ .
Explore different support sets with size  $\tilde{K}$ : Using the blocking method mentioned above, we can approximate the Rashomon set for many different support sets that are subsets of the original support set. The approximation of the ellipsoid for a smaller support set is more accurate when the size difference between the original support set and the new support set is small. Therefore, instead of using a very large support set at the beginning, we use a sparse GAM algorithm such as FastSparse [12] with a relatively weak sparsity penalty to get a support set  $S$  whose size  $K$  is moderately larger
than the size of support sets we want to explore, but covers all bins we want to potentially merge to form a new support set. For simplicity of defining the loss threshold of the Rashomon set, here we consider only support sets with size  $\tilde{K}$ . If we want to explore different  $\tilde{K}$ , we can repeat this process. Suppose our goal is to explore the Rashomon set with loss threshold  $\delta$ . The first step is to use the methods in Section 3.1 to obtain the ellipsoid  $\hat{R}$  approximating the  $\theta$ -Rashomon set on the original support set, where  $\theta = \delta + \lambda_0 (K - \tilde{K})$ . Then we can enumerate all possible ways  $(\binom{K-p}{K-\tilde{K}}$  ways in total) to merge bins in the original support set to get any subset  $\tilde{S}$  whose size is  $\tilde{K}$ , and calculate the intersection (i.e., ellipsoid)  $R_{P_{\tilde{S}}} := P_{\tilde{S}} \cap \hat{R}$  between  $\hat{R}$  and the hyperplane  $P_{\tilde{S}}$  corresponding to merging  $S$  into  $\tilde{S}$ . All nonempty  $R_{P_{\tilde{S}}}$  are valid approximations of the Rashomon set.
4 Applications of the Rashomon Set
We now show four practical challenges related to GAMs that practitioners can now solve easily using the approximated Rashomon sets.
4.1 Variable Importance within the Model Class
Variable importance can be easily measured for GAMs. For example, Explainable Boosting Machines [8, 27] measures the importance of each variable by the weighted mean absolute coefficient value of bins corresponding to this variable, i.e.,  $VI(\mathbf{x}_{-,j}) = \sum_{k=0}^{B_j-1} \pi_{j,k} |w_{j,k}|$ . This measure is based on one model, but given the Rashomon set, we can now provide a more holistic view of how important a variable is by calculating the range of variable importance among many well-performing models; this is called the Model Class Reliance [14]. Specifically,  $VI_{-}$  and  $VI_{+}$  are the lower and upper bounds of this range, respectively. For example, the range of importance of feature  $j$  is defined as
$$
\left[ V I _ {-} \left(\mathbf {x} _ {\cdot , j}\right), V I _ {+} \left(\mathbf {x} _ {\cdot , j}\right) \right] = \left[ \min  _ {\boldsymbol {\omega} \in R (\theta)} V I \left(\mathbf {x} _ {\cdot , j}\right), \max  _ {\boldsymbol {\omega} \in R (\theta)} V I \left(\mathbf {x} _ {\cdot , j}\right) \right]. \tag {8}
$$
A feature with a large  $VI_{-}$  is important in all well-performing models; a feature with a small  $VI_{+}$  is unimportant to every well-performing model. We use  $\hat{R}$  to approximate  $R(\theta)$  and estimate  $VI_{-}$  by solving a linear programming problem with a quadratic constraint, and we estimate  $VI_{+}$  by solving a mixed-integer programming problem. Solving these problems gives a comprehensive view of variable importance for the problem, not just a single model.
Lower bound of variable importance:  $VI_{-}$  of feature  $j$  can be obtained by solving the following linear programming problem.
$$
\min  _ {\left[ \omega_ {j, 0}, \dots , \omega_ {j, B _ {j} - 1} \right]} \sum_ {k = 0} ^ {B _ {j} - 1} \pi_ {j, k} \left| \omega_ {j, k} \right| \quad \text {s . t .} \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) ^ {T} \mathbf {Q} (\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}) \leq 1. \tag {9}
$$
Since  $\pi_{j,k} \geq 0$ , Problem (9) can be solved using an LP solver. The objective minimizes variable importance, and the constraint ensures the solution is in the Rashomon set.
Upper bound of variable importance: The maximization problem cannot be solved through linear programming since  $[\omega_{j,0},\dots,\omega_{j,B_j - 1}]$  can be arbitrarily large. Instead, we formulate it as a mixed-integer program. Let  $M$  be a relatively large number (e.g., 200 is large enough for real applications (see Appendix E)) and let  $I_{k}$  be a binary variable.
$$
\max  _ {\left[ \omega_ {j, 0} ^ {\prime}, \dots , \omega_ {j, B _ {j} - 1} ^ {\prime}, I _ {1}, \dots , I _ {B _ {j} - 1} \right]} \sum_ {k = 0} ^ {B _ {j} - 1} \pi_ {j, k} \omega_ {j, k} ^ {\prime}
$$
$$
\text {s . t .} \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) ^ {T} \mathbf {Q} \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) \leq 1 \tag {10}
$$
$$
\omega_ {j, k} + M \times I _ {k} \geq \omega_ {j, k} ^ {\prime}, - \omega_ {j, k} + M \times (1 - I _ {k}) \geq \omega_ {j, k} ^ {\prime}, \forall k \in \{0, \dots , B _ {j} - 1 \}
$$
$$
\omega_ {j, k} \leq \omega_ {j, k} ^ {\prime}, - \omega_ {j, k} \leq \omega_ {j, k} ^ {\prime}, \forall k \in \{0,..., B _ {j} - 1 \}.
$$
The last two constraints ensure that  $\omega_{j,k}^{\prime}$  is defined as the absolute value of  $\omega_{j,k}$ . However, solving mixed-integer programs is usually time-consuming. We propose another way to solve the maximization problem. Since  $\omega_{j,k}$  can be either positive or negative, we enumerate all positive-negative combinations for  $\omega_{j,k}, k \in \{0, \dots, B_j - 1\}$ , solve the LP problem with the sign constraint enforced, and choose the maximum value (see Algorithm 2 in the Appendix E).
4.2 Monotonic Constraints