SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A use-case for the GAM Rashomon set is finding an accurate model that satisfies monotonicity constraints. This can be obtained by solving a quadratic programming problem. For example, if a user wants  $f_{j}(\mathbf{x}_{j})$  to be monotonically increasing, the optimization problem can be formalized as:
$$
\min  _ {\boldsymbol {\omega}} \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) ^ {T} \mathbf {Q} \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) \quad \text {s . t .} \quad \omega_ {j, k} \leq \omega_ {j, k + 1}, k \in \{0, \dots , B _ {j} - 1 \}, \tag {11}
$$
where we check that the solution is  $\leq 1$ ; i.e., that solution  $\omega$  is in the approximated Rashomon set.
4.3 Find robustly occurring sudden jumps in the shape functions
Sudden changes (e.g., a jump or a spike) in the shape functions of GAMs are known to be useful signals for knowledge discovery [10, 11] and debugging the dataset [28]. However, due to the existence of many almost equally good models that have different shape functions, it is hard to identify if a change is a true pattern in the dataset or just a random artifact of model fitting. However, with the approximated Rashomon set, we can calculate the proportion of models that have such a change among all near-optimal models and visualize the shape functions (see Section 5.4).
4.4 Find the model within the Rashomon set closest to the shape function requested by users
As discussed, the main benefit of the Rashomon set is to provide users a choice between equally-good models. After seeing a GAM's shape functions, the user might want to make edits directly, but their edits can produce a model outside of the Rashomon set. We thus provide a formulation that projects back into the Rashomon set, producing a model within the Rashomon set that follows the users' preferences as closely as possible. We find this model by solving a quadratic programming problem with a quadratic constraint. Let  $\omega_{req}$  be the coefficient vector that the user requests. The problem can be formulated as follows:
$$
\min  _ {\boldsymbol {\omega}} \left\| \boldsymbol {\omega} - \boldsymbol {\omega} _ {r e q} \right\| _ {2} ^ {2} \quad \text {s . t .} \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) ^ {T} \mathbf {Q} \left(\boldsymbol {\omega} - \boldsymbol {\omega} _ {c}\right) \leq 1. \tag {12}
$$
Problem (12) can be easily solved through a quadratic programming solver.
Our evaluation answers the following: 1. How well does our method approximate the Rashomon set? 2. How does the approximated Rashomon set help us understand the range of variable importance in real datasets? 3. How well does the approximated Rashomon set deliver a model that satisfies users' requirements? 4. How does the Rashomon set help us investigate changes in the shape function?
We use four datasets: a recidivism dataset (COMPAS) [29], the Fair Isaac (FICO) credit risk dataset [30], a Diabetes dataset [31], and an ICU dataset MIMIC-II [32].
5.1 Precision and volume of the approximated Rashomon set
As we mentioned in Section 3, our goal is to create an approximation of the Rashomon set whose volume is as large as possible, while ensuring that most points in the approximated Rashomon set are within the true Rashomon set. To measure precision of our approximation, models within and outside the true Rashomon set  $R(\theta)$  are considered as "positive" and "negative" respectively; models within and outside the approximated Rashomon set  $\hat{R}$  are considered as "predicted positive/negative." Accordingly, the precision is defined as the proportion of points  $(\omega, \omega_0)$ , equally weighted and within  $\hat{R}$ , having  $\mathcal{L}(\omega, \omega_0) \leq \theta$ , i.e., they are in  $R(\theta)$ . The recall is defined as the proportion of points in  $R(\theta)$  that are also in  $\hat{R}$ . Note that recall  $= \frac{\operatorname{Vol}(\hat{R} \cap R(\theta))}{\operatorname{Vol}(R(\theta))} = \operatorname{precision} \times \frac{\operatorname{Vol}(\hat{R})}{\operatorname{Vol}(R(\theta))}$ . Since  $\operatorname{Vol}(R(\theta))$  is a constant, given the same volume  $\operatorname{Vol}(\hat{R})$ , recall is proportional to precision.
We will develop some natural baselines to compare to. For fair comparison, we rescale the Rashomon set approximated by baselines to have the same volume as our  $\hat{R}$ . Then, we sample 10,000 points from our  $\hat{R}$  and the rescaled baseline Rashomon sets, calculate their loss, and estimate precision.
Figure 1: Precision versus the size of the approximated Rashomon set.
One baseline is using our Hessian initialization of  $\mathbf{Q}$ , i.e.,  $\mathbf{H} / (2(\theta - \mathcal{L}(\boldsymbol{\omega}^{*})))$ , denoted as "hessian" in our comparison. Another baseline is to use the identity matrix to approximate  $\mathbf{Q}$ , i.e., the approximated set is a sphere. Another type of baseline comes from fitting GAMs on bootstrap sampled subsets of data. For a fixed support set, we calculate coefficient vectors from two different methods (logistic regression and EBMs [8]) on many subsets of data. This approach samples only a finite set of coefficient vectors rather than producing a closed infinite set such as an ellipsoid, and therefore its volume is not measurable. Thus, we use optimization to find the minimum volume ellipsoid that covers most coefficient vectors. This is the MVEE problem [33]; more details are provided in Appendix C. Sampling from the posterior distribution of a Bayesian model is also a way to get many different GAMs. We tried Bayesian logistic regression (using Hamiltonian Monte Carlo), but it is too slow ( $\geq 6$  hours) to converge and produce enough samples to construct the minimum volume ellipsoid. So we did not include this baseline.
Figure 1 compares the precision and volume of our Rashomon set with baselines. Figure 1a shows that when the volume of the approximated Rashomon sets are the same between methods, our Rashomon set has better precision than the baselines. In other words, the Rashomon set approximated by our methods has the largest intersection with the true Rashomon set. The results for the hessian is worse than ours but better than other baselines, which means our proposed initialization is already better than other baselines. Also, as  $\theta$  becomes larger, the hessian method becomes better and sometimes close to the result after optimization. Logistic regression and EBM with bootstrapping do not achieve good precision, because coefficient vectors trained by these methods on many subsets of data can be too similar to each other, leading to ellipsoids with defective shapes that do not match the true Rashomon set boundary even after rescaling.
In general, shrinking the approximated Rashomon set leads to higher precision and expanding it leads to better recall. Figure 1b shows the performance of different methods under this tradeoff. Our method still dominates others even when we rescaled the volume with different factors.
The previous experiments evaluate the performance when the support set is fixed. To explore many different support sets, we use the blocking method (Method 2) from Section 3.2. Method 2 is much faster than Method 1, so we need to check that it performs similarly, according to
Figure 2: Box plots of the volume ratio and precision ratio of Method 1 over Method 2 across different  $\tilde{K}$ , for two datasets. It shows the precision and volume ratios are close to 1, indicating that Method 2 performs similarly to Method 1.
both the volume and precision of the approximated Rashomon set. The volume ratio, defined by  $\sqrt[K]{\mathrm{Vol}(\hat{R}_{\mathrm{method1}}) / \mathrm{Vol}(\hat{R}_{\mathrm{method2}})}$ , should be 1 if the two methods perform similarly. The precision ratio, the precision of Method 1 over the precision of Method 2, again should be close to 1 if the methods perform similarly.
Figure 2 shows the precision ratio and volume ratio of 100 different new support sets where  $u > 0$ , i.e., the approximated Rashomon set is non-empty. Clearly, Method 2 will break down if we choose the support size  $\tilde{K}$  to be too small. The results from Figure 2 indicate what we expected: precision and volume ratios are close to 1, even for relatively small  $\tilde{K}$ . Importantly, Method 2 is much more efficient than Method 1. For example, the running time is  $< 0.001$  seconds for Method 2 but ranges from 300 to 1100 seconds for Method 1 on all three datasets (COMPAS, FICO, MIMIC-II). More results are in Appendix C.
(a) Variable importance range of the Diabetes dataset.
Figure 3: (a) Variable importance range of the Diabetes dataset and (b) shape functions of "Glucose." "Not fix": the shape function is obtained by solving Problem (10). "Fix": coefficients of all other features except "Glucose" are set to the same values as in  $\omega_{c}$ . ( $\lambda_{s} = 0.001$ ,  $\lambda_{2} = 0.001$ ,  $\theta = 1.01\mathcal{L}^{*}$ )
(b) Shape functions of "Glucose."
5.2 Variable importance range
We solve Problem (9) and (10) to bound the variable importance; results on the Diabetes dataset are shown by dark blue segments in Figure 3a. The light blue segments show the range with an additional constraint that coefficients of features other than the one we are interested in are fixed. Adding this extra constraint leads to lower  $VI_{+}$  but has negligible impacts on  $VI_{-}$ . The "Glucose" feature has dominant  $VI_{-}$  and  $VI_{+}$  compared with other features, indicating that for all well-performing GAMs, this feature is the most important. In fact, the large  $VI_{-}$  indicates that Glucose remains important even for the model that relies least on it in the Rashomon set. This makes sense, because an oral glucose tolerance test indicates diabetes if the plasma glucose concentration is greater than two hours into the test. Features "BMI" and "Age" also have high  $VI_{-}$ , indicating both of them are more important than other features.
(a) Monotonically decreasing
(b)User request case 1
Figure 4: Example shape functions of "BloodPressure" that satisfy users requirement. (a) The shape function of "BloodPressure" is desired to be monotonically decreasing. But it has a jump at BloodPressure  $\sim 60$ . The optimization time to find the monotonically decreasing shape function is 0.04 seconds. (b) Removes the jump by connecting to the right step. The optimization problem is solved in 0.0024 seconds. (c) Removes the jump by connecting to the left step. The time used to get the model is 0.0022 seconds. ( $\lambda_0 = 0.001$ ,  $\lambda_2 = 0.001$ ,  $\theta = 1.01\mathcal{L}^*$ ).
(c)User request case 2
Figure 5: Different shape functions (in red) of "PFratio" in MIMIC-II sampled from  $\hat{R}$ .
Figure 3b shows shape functions of  $\omega_{c}$  and the coefficient vectors obtained when the lowest and highest variable importance are achieved for "Glucose." All these shape functions follow the same almost-monotonic trend, which means the higher "Glucose" level, the more likely it is that a patient has diabetes. The variable tends to have higher variable importance when it has a larger range in the predicted logit. Shape functions based on  $\omega_{VI_{+}}$  without fixing other coefficients can deviate more from  $\omega_{c}$ ; when we do not fix other coefficients, there is more flexibility in the shape functions.
5.3 User preferred shape functions
A significant advantage of the Rashomon set is that it provides the opportunity for users to contribute their domain knowledge flexibly without retraining the model. Note that all models within the approximated Rashomon set predict equally well out of sample (see Table 7 in Appendix), so the user can choose any of them. We use the "BloodPressure" feature in the Diabetes dataset as an example. Figure 4 shows the shape function of "BloodPressure" (in gray), in which a jump occurs when blood pressure is around 60. A potential user request might be to make the shape function monotonically decreasing. By solving Problem (11) with the constraint on coefficients related to "BloodPressure," we get the shape function colored in yellow in Figure 4a. The jump is reduced by dragging up the left step. Suppose a user prefers to remove the jump by elevating the left step (see inset plot in Figure 4b). By solving Problem (12), we find that the specified shape function is not within the Rashomon set, and we find the closest solution in green, which still has a small jump at 60. Another user might prefer to remove the jump by forcing both left and right steps at the same coefficients (see inset plot in Figure 4c). This specified shape function is also outside the Rashomon set, and the closest solution keeps the most steps as required with a small step up. Different user-specified shape functions lead to different solutions. The approximated Rashomon set can thus provide a computationally efficient way for users to find models that agree with both their expectations and the data. Our work could be integrated with tools such as GAMChanger [34] which allows users to directly edit a GAM, but there is no guarantee that the resulting model still performs well.
5.4 Changes in shape functions
When we see a jump in the shape function, we might want to know whether this jump often exists in other well-performing models. Using the approximated Rashomon set, we can give an answer. For example, the shape function of "PFratio" in the MIMIC-II dataset has a sudden dip around 330 (see the gray curve in Figure 5). The jump is so deep that we might wonder if such a jump is present in other well-performing GAMs. We sample 10,000 points from our  $\hat{R}$  and find that 7012 samples have a downward jump at the same position. We are surprised to find that in some of the remaining 2988 samples, the jump is instead upward. Figure 5 shows four different shape functions of "PFratio" sampled from  $\hat{R}$ . The dramatic magnitude change in either direction could be caused by the small number of data points falling in this bin ( $\pi_{\mathrm{bin}} = 1.2\mathrm{e - }4$ ). Since the weight is so small, even a tremendous change leads to only a small impact on the loss. In practice, using our proposed optimization methods in Section 4, users can reduce or amplify the jump in the shape function or even create a different curve, as in Figure 4. Note that our ability to find a diverse set of shape functions illustrates an advantage of the Rashomon set that cannot be easily seen with other approaches.
Our work approximates the Rashomon set of GAMs in a user-friendly way. It enables users to explore, visualize, modify, and gain insight from GAM shape functions. Our work represents a paradigm shift in machine learning, enabling unprecedented flexibility for domain experts to interact with models without compromising performance. Our contributions open a door for enhancing human-model interaction through using the new toolkit provided by the Rashomon set.
One limitation that could be creatively considered in future work is that there are numerous possible support sets, leading to the question of how a human might comprehend them. A carefully-designed interactive visual display might be considered for this task.
We thank the anonymous reviewers for their suggestions and insightful questions. We gratefully acknowledge support from grants DOE DE-SC0023194, NIH/NIDA R01 DA054994, NSF IIS-2130250, NSF DGE-2022040, and DOE DE-SC0021358. We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC). Nous remercions le Conseil de recherches en sciences naturelles et en génie du Canada (CRSNG) de son soutien.
Implementations of our methods is available at https://github.com/chudizhong/GAMsRashomonSet.
[1] Leo Breiman. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical Science, 16(3):199-231, 2001.
[2] Rui Xin, Chudi Zhong, Zhi Chen, Takuya Takagi, Margo Seltzer, and Cynthia Rudin. Exploring the whole rashomon set of sparse decision trees. Advances in Neural Information Processing Systems, 35:14071-14084, 2022.
[3] Lesia Semenova, Cynthia Rudin, and Ronald Parr. On the existence of simpler machine learning models. In ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT), 2022.
[4] Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al. Underspecification presents challenges for credibility in modern machine learning. Journal of Machine Learning Research, 2020.
[24] Theja Tulabandhula and Cynthia Rudin. Robust optimization using machine learning for uncertainty sets. In International Symposium on Artificial Intelligence and Mathematics, ISAIM 2014, Fort Lauderdale, FL, USA, January 6-8, 2014, 2014.
[25] Beau Coker, Cynthia Rudin, and Gary King. A theory of statistical inference for ensuring the robustness of scientific results. Management Science, 67(10):6174-6197, 2021.
[26] Jiayun Dong and Cynthia Rudin. Exploring the cloud of variable importance for the set of all good models. Nature Machine Intelligence, 2(12):810-824, 2020.
[27] Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. Interpretml: A unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223, 2019.
[28] Zhi Chen, Sarah Tan, Harsha Nori, Kori Inkpen, Yin Lou, and Rich Caruana. Using explainable boosting machines (ebms) to detect common flaws in data. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 534-551. Springer, 2021.
[29] J. Larson, S. Mattu, L. Kirchner, and J. Angwin. How we analyzed the COMPAS recidivism algorithm. ProPublica, 2016.
[30] FICO, Google, Imperial College London, MIT, University of Oxford, UC Irvine, and UC Berkeley. Explainable Machine Learning Challenge. https://community.fico.com/s/explainable-machine-learning-challenge, 2018.
[31] Jack W Smith, James E Everhart, WC Dickson, William C Knowler, and Robert Scott Johannes. Using the adap learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the annual symposium on computer application in medical care, page 261. American Medical Informatics Association, 1988.
[32] Mohammed Saeed, Christine Lieu, Greg Raber, and Roger G Mark. Mimic ii: a massive temporal icu patient database to support research in intelligent patient monitoring. In Computers in Cardiology, pages 641-644. IEEE, 2002.
[33] Michael J Todd and E Alper Yildirim. On khachiyan's algorithm for the computation of minimum-volume enclosing ellipsoids. Discrete Applied Mathematics, 155(13):1731-1744, 2007.
[34] Zijie J Wang, Alex Kale, Harsha Nori, Peter Stella, Mark Nunnally, Duen Horng Chau, Michaela Vorvoreanu, Jennifer Wortman Vaughan, and Rich Caruana. Gam changer: Editing generalized additive models with interactive visualization. Advances in Neural Information Processing Systems, Bridging the Gap: From Machine Learning Research to Clinical Practice (Research2Clinics) Workshop, 2021.
[35] Steven Diamond and Stephen Boyd. CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research, 17(83):1-5, 2016.
[36] Akshay Agrawal, Robin Verschueren, Steven Diamond, and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42-60, 2018.
A Summary of datasets
We show experimental results on four datasets: a recidivism dataset (COMPAS) [29], the Fair Isaac (FICO) credit risk dataset [30] used for the Explainable ML Challenge, the Diabetes dataset [31], and an ICU dataset MIMIC-II [32]. Table 1 summarizes all the datasets. We note that these are real-world datasets using features that are known to be used in systems of this type. Specifically, the COMPAS dataset is a criminal recidivism dataset and we use features that are based on criminal history, age, etc. For FICO, the data were provided by FICO itself based on how credit scores are usually constructed. MIMIC-II is based on features observed in ICU patients that are predictive of death, and the Diabetes dataset also uses typical measurements taken during pregnancy that are indicative of diabetes. The features for each dataset are generally available for the decision and they are interpretable.
Dataset	Samples	Features	Classification task
COMPAS	6907	7	Predict if someone will be arrested ≤ 2 years of release
FICO	10459	23	Predict if someone will default on a loan
Diabetes	768	9	Predict whether a pregnant woman has diabetes
MIMIC-II	24508	17	Predict whether a patient dies in the ICU
Table 1: Summary of datasets
B Sampling Uniformly from Ellipsoid
Algorithm 1 SampleFromEllipsoid  $(\mathbf{Q},\omega_c)$
Input: parameters of the ellipsoid  $\mathbf{Q} \in \mathbb{R}^{m \times m}$ ,  $\omega_{c} \in \mathbb{R}^{m}$
Output: a point inside the ellipsoid  $\omega \in \mathbb{R}^m$
1:  $\mathbf{u} \sim \mathcal{N}(\mathbf{0},\mathbf{I})$  // sample an  $m$  dimensional vector from standard multivariate Gaussian distribution
2:  $\mathbf{u} \gets \mathbf{u} / \| \mathbf{u}\|_2$  // normalize it to get a unit-vector
3:  $r \sim U(0,1) //$  get a sample from uniform distribution
4:  $r \gets r^{1/m}$  // rescale to get the radius of a sample in a unit sphere
5:  $\mathbf{y} \gets r\mathbf{u} // \mathbf{y}$  is a random point inside a unit sphere
6:  $\Lambda, \mathbf{V} = \operatorname{Eig}(\mathbf{Q})$  // Eigen-decomposition, diagonal of  $\Lambda$  are the eigenvalues, and columns of  $\mathbf{V}$  are eigenvectors
7:  $\mathbf{x} \gets \boldsymbol{\Lambda}^{-\frac{1}{2}} \mathbf{V}^T \mathbf{y} \parallel$  get a point in the ellipsoid  $\mathbf{x}^T \mathbf{Q} \mathbf{x} \leq 1$
8:  $\omega = \mathbf{x} + \omega_{c}$  // shift it so that the center is  $\omega_{c}$
return  $\omega$
Algorithm 1 describes the algorithm to uniformly sample from the ellipsoid  $\{\pmb {\omega}\in \mathbb{R}^m:(\pmb {\omega} - \pmb {\omega}_c)^T\mathbf{Q}(\pmb {\omega} - \pmb {\omega}_c)\leq 1\}$ . The algorithm first samples a random point inside a high dimensional unit sphere (line 1-5), and applies a linear transformation (calculated from  $\mathbf{Q}$  and  $\pmb{\omega}_{c}$ ) to get the point in the target ellipsoid (line 6-8). The whole process can be decomposed into a purely stochastic part, i.e., sample in the unit sphere, and a deterministic part, which is differentiable. Using this sampling algorithm, we can get samples for objective (6) and use gradient-based methods to optimize  $\mathbf{Q}$  and  $\pmb{\omega}_{c}$ . In addition, the algorithm is also used to sample data for the problem in Section 4.3 and to calculate precisions in Section 5.1.
C Precision and volume of the approximated Rashomon set
To run our method, we set the learning rate to 0.0001 and run 1000 iterations.  $C$  is set to 500. For the logistic regression and EBM baselines, we sample 2000 coefficient vectors by fitting GAMs on the bootstrap sampled subsets of data. We run logistic regression with  $\ell_2$  penalty and EBM with no interaction terms. We then find the minimum volume ellipsoid that can cover most coefficient vectors. Given a set of coefficient vectors  $\omega_{\mathrm{samples}}$ , we solve the following problem:
$$
\min  _ {\mathbf {Q}, \boldsymbol {\omega} _ {c}} - \det  (\mathbf {Q}) ^ {\frac {1}{2 m}} + \xi \cdot \frac {1}{2 0 0 0} \sum_ {i = 1} ^ {2 0 0 0} \max  \left(\left\| \mathbf {Q} ^ {1 / 2} \left(\boldsymbol {\omega} _ {\text {s a m p l e} _ {i}} - \boldsymbol {\omega} _ {c}\right) \right\| ^ {2} - 1, 0\right). \tag {13}
$$
We solve this problem via gradient descent.  $\xi$  is set to 1000. The number of iterations and learning rate in GD are set to 1000 and 0.01, respectively. We initialize  $\mathbf{Q}$  by the ZCA whitening matrix and  $\omega_{c}$  by the average of  $\omega_{\mathrm{samples}}$ .
After rescaling the Rashomon set approximated by baselines, we sample 10,000 points from our  $\hat{R}$  and rescaled baseline Rashomon sets, calculate the loss, and get the precision. We include more figures in this appendix that are similar to Figure 1 to compare with baselines using different values of  $\lambda_{s}$  and  $\theta$ .
$$
\lambda_ {s} = 0. 0 0 0 5
$$
$$
\lambda_ {s} = 0. 0 0 1
$$
$$
\lambda_ {s} = 0. 0 0 1 5
$$