Automatic Outlier Rectification via Optimal Transport
Authors: Jose Blanchet
Jiajin Li
Markus Pelger
Greg Zanotti
Venue: NeurIPS 2024 poster (2024)
Keywords: Outlier Rectification; Optimal Transport; Statistically Robust
Primary area: other
TL;DR: This paper introduces a novel framework for outlier detection using optimal transport with a concave cost function, integrating outlier rectification and estimation into a single optimization process.

Abstract: In this paper, we propose a novel conceptual framework to detect outliers using optimal transport with a concave cost function. Conventional outlier detection approaches typically use a two-stage procedure: first, outliers are detected and removed, and then estimation is performed on the cleaned data. However, this approach does not inform outlier removal with the estimation task, leaving room for improvement. To address this limitation, we propose an automatic outlier rectification mechanism that integrates rectification and estimation within a joint optimization framework. We take the first step to utilize the optimal transport distance with a concave cost function to construct a rectification set in the space of probability distributions. Then, we select the best distribution within the rectification set to perform the estimation task. Notably, the concave cost function we introduced in this paper is the key to making our estimator effectively identify the outlier during the optimization process. We demonstrate the effectiveness of our approach over conventional approaches in simulations and empirical analyses for mean estimation, least absolute regression, and the fitting of option implied volatility surfaces.

Automatic Outlier Rectification via Optimal Transport
Jose Blanchet
Jiajin Li Sauder School Business University of British Columbia jiajin. li@sauderr.... ubc.ca
Dept. of Management Science & Engineering Stanford University
jose. blanchet@stannford
Markus Pelger
Greg Zanotti
Dept. of Management Science & Engineering Stanford University
Dept. of Management Science & Engineering Stanford University
mpelger@stanford.cmm
granotti@stanford.edd
Abstract
In this paper, we propose a novel conceptual framework to detect outliers using optimal transport with concave cost function. Conventional outlier detection approaches typically use two-stage procedure: first, outliers are detected and removed, and then estimation performed on the cleaned data. However, this approach does not inform outlier removal with the estimation task, leaving room for improvement. To address this limitation, we propose an automatic outlier rectification mechanism that integrates rectification and estimation within joint optimization framework. We take first step to utilize the optimal transport distance with a concave cost function to construct rectification set in the space probability distributions. Then, we select the best distribution within the rec- tification set to perform the estimation task. Notably, the concave cost function we introduced in this paper is the key to making our estimator effectively identify the outlier during the optimization process. We demonstrate the effectiveness of our approach over conventional approaches in simulations and empirical analyses for mean estimation, least absolute regression, and the fitting of option implied volatility surfaces.
Introduction
Outlier removal has a long tradition in statistics, both in theory and practice. This is because is common to have (for example, due to collection errors) data contamination or corruption. Directly applying learning algorithm to corrupted data can, naturally, lead undesirable out-of-sample performance. Our goal in this paper is to provide a single-step optimization mechanism based on optimal transport for automatically removing outliers.
The challenge of outlier removal has been documented for centuries: early work is introduced in e.g. Gergonne [1821] and Peirce [1852]. Yet the outlier removal problem continues interest practitioners and researchers alike due to the danger of distorted model estimation. natural family of approaches followed in the literature takes the form of "two-stage" methods, which involve outlier removal step followed by an estimation step. Methods within this family range from rules of thumb, such removing outliers beyond a particular threshold based robust measure of scale like the interquartile range [Tukey, 1977], to various model-based or parametric distributional assumption-baat based tests [Thompson, 1985]. While the two-stage approach can be useful by separating the estimation task from the outlier removal objective, it is not without potential pitfalls. For example, outlier detection which relies on specific fitted model may tend overfit to a particular type of outlier, which can mask the effect of other types [Rousseeuw and Leroy, 1987]. Conversely, the
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

outlier detection step may be overly conservative if it is not informed by the downstream estimation task; this can lead significant reduction in the number of observations available for model estimation in the next step, resulting in a loss of statistical efficiency [He and Portnoy, 1992].
Robust statistics, pioneered by Box [1953], Tukey [1960], Huber [1964] and others such as Hampel [1968, 1971] offers alternative approaches for obtaining statistical estimators in the presence of out- liers without removing them, particularly in parametric estimation problems such as linear regression. Beyond these, one closely related approach is the minimum distance iunetionals-based estimator, introduced by Parr and Schucany [1980], Millar [1981], Donoho and Liu [1988a.b], Park et al. [1995], Zhu et al. [2022], Jaenada et al. [2022]. This method involves projecting the corrupted distribution onto a family of distributions using distribution metric and selecting the optimal estimator for the resulting distribution. However, this projection mechanism is not informed by the estimation task (such as fitting a linear regression or neural network). Thus, without additional information about the contamination model or estimation task, it can be challenging to choose an appropriate family of distributions for projection, which may lead to limitations similar to those outlined above in practice.
In this paper, we propose novel approach to integrate both outlier detection and estimation in a joint optimization framework. A key observation is that statisticians aim to "clean" data before decisions are made; thus, ideal robust statistical estimators tend to be optimistic. To address this, we introduce the rectification set: a ball centered around the empirical (contaminated) distribution and defined by the optimal transport distance (see Villani [2009], Peyré et al. [2019]) in probability space. This rectification set aims to exclude potential outliers and capture the true underlying distribution. allowing us to minimize the expectation over the "best-case" scenarios, leading to min-min problem. The study in Jiang and Xie [2024] also explores connections with min-min-type problems, but concentrates on the construction of artificially constructed rectification sets to cover certain existing estimators from the robust statistics literature. However, our primary focus is on introducing a novel rectification set, which is based on the optimal transport approach with a concave cost function.
To automatically detect outliers during the estimation process, one of our main contributions is the use of concave cost function for the optimal transport distance. This function encourages what we refer to as 'long haul" transportation, in which the optimal transport plan moves only a portion of the data to distant location, while leaving other parts unchanged. This strategic approach effectively repositions identified outliers closer to positions that better align with the clean data. Our novel formulation then involves minimizing the expected loss under the optimally rectified empirical distribution. This rectification is executed through the application of an optimal transport distance with concave cost function, thereby correcting outliers to enhance performance within fixed estimation task. More importantly, our method distinguishes itself from distributionally robust optimization (DRO) [Ben-Tal et al., 2013, Bayraksan and Love, 2015, Wiesemann et al., 2014, Delage and Ye, 2010, Gao and Kleywegt, 2022, Blanchet and Kang. 2017, Shafieezadeh Abadeh et al., 2015, Shaficezadeh-Abieee correction mechanisms initiated by the robust formulation employing a min-min strategy.
et al., 2019, Sinha et al., 2018, Kuhn al., 2019] due to the distinctive
As we discuss in Section 2 below, the timing of error generation differs crucially between DRO and robust statistics. DRO approach employs a min-max game strategy to control the worst-case loss over potential post-decision distributional shifts. In contrast, the robust estimator acts after the pre-decision distributional contamination materializes. Thus the approach of robust statistics can be motivated being closer to a max-min game against nature. As a consequence, in robust statistics, the adversary moves first, and therefore the statistician can be more optimistic that they can rectify the contamination applied by the nature thus motivating the min-min strategy suggested above. However, it is also worth mentioning that Kang and Bansal [2023]. Nietert et al. [2023] are also trying to formulate the outlier-robust problem in a min max-max form, incorporating additional information about the contamination model to further shrink the ambiguity set.
It is useful to contrast our approach with more traditional approaches to achieving robustness in statistics, such as M-estimators and random sample consensus methods such as RANSAC. The class of M-estimators [Lehmann and Casella, 2006, Van der Vaart, 2000, Hayashi, 2000] is quite broad. This class includes methods which are both robust and not robust. M-estimators in the context of traditional robust statistics (as in introduced by Huber in the 1960s) attempt to achieve robustness by using specific loss functions which make the estimation procedure less sensitive to outliers. Common examples include the Huber loss function and least absolute deviations regression. Our approach is more fundamental as it builds on top of a given (general) loss and the statistician has full control of
2

---

the "outlier removal" modeling task (via the specification of optimal transport theory using concave optimal transport cost) and the statistical task (via the chosen loss). In addition, the statistician also obtains the optimal "rectifier" (i.e. transporter) while the amount of rectification can be estimated via a cross-validation approach. Our approach thus conceptually different from M-estimators. The benefits of our approach over M estimators (including the Huber loss) are illustrated in the experimental sections of our paper. RANSAC, first proposed in Fischler and Bolles [1981], handles outliers by iteratively selecting random data subsets, fitting models, and evaluating models on inliers to reach a best model. In contrast, our method optimally selects the set of points for rectification using single optimization procedure. Though work random sample consensus methods which guide sampling in more intelligent ways, such as Chin et al. [2011], may be seen as connected to our approach, our work is notably different. For example, random sample consensus methods only identify inliers, while the approach proposed in this paper identifies inliers and outliers and rectifies outliers. Our approach scales to high dimensions, while RANSAC encounters fundamental issues with sampling in such dimensions. Finally, our approach is based on optimal transport theory rather than heuristic or random sampling-
We summarize our main contributions as follows:
(i) Novel statistically robust estimator. We propose a new statistically robust estimator that incorporates novel rectification set constructed using the optimal transport distance. By employing a concave cost function within the optimal transport distance, our estimator enables automatic outlier rectification. We prove that the optimal rectified distribution can be found via a finite convex program, and show that we can determine the optimal rectification set uncertainty budget via cross-validation (ii) Connection to adaptive quantile regression. For mean estimation and least absolute regression, demonstrate that our robust estimator is equivalent to an adaptive quantile (regression) estimator, with the quantile controlled by the budget parameter 8. Furthermore, we prove that the optimal rectified distribution exhibits a long haul structure that facilitates outlier detection.
(iii) Effectiveness in estimating the option implied volatility surface. We evaluate estimator on various tasks, including mean estimation, least absolute regression, and option implied volatility surface estimation. Our experimental results demonstrate that our estimator produces surfaces that are 30.4% smoother compared to baseline estimators, indicating success in outlier removal, as surfaces should be smooth based on structural financial properties. Additionally, it achieves an average reduction of 6.3% in mean average percent error (MAPE) across all estimated surfaces, providing empirical evidence of the effectiveness of our rectifying optimal transporter.
2 DRO and Robust Statistics as Adversarial Games
In this section, summarize conceptually how robust statistics is different from DRO (for more details, we refer the reader to e.g. Blanchet et al. [2024]). To lay a solid mathematical foundation, we begin by investigating a generic stochastic optimization problem. Here, assume that Z is a random vector in a space Z C Rd that follows the distribution P... The set of feasible model parameters is denoted θ (assumed to be finite-dimensional to simplify). Given a realization = and a model parameter 0 E θ the corresponding loss is E(0,2) 2). A standard expected loss minimization decision rule is obtained by solving
min Ep, [((0,E)] / ((0,))P..C)
deB
Since IP. generally unknown, to approximate the objective function in (1), we often gather n independent and identically distributed (i.i.d.) samples {=)]², from the unknown data-generating distribution IP, and consider the empirical risk minimization counterpart,
1 Σ ((0,4), n
min E, [)( Ice
where IP,, denotes the empirical measure Σm= 52, and 6_ is the Dirac measure centered at z. These problems are solved within the context of the general data-driven decision-making cycle below:
In the cycle depicted in Figure 1, we usually collect i.i.d samples from the unknown data-generating distribution P', which may be identical or distinct from the clean distribution P.. Subsequently, we make a decision (e.g., parameter estimation) based on model, P built from these samples. Such model could be parametric or nan-paramrrric. These decisions are then put into action within the out-of-sample environment P. which may or may not conform to the distribution P.. In this general cycle, the sample average method (2) may lead to poor out-of guarantees. This motivates
3

---

Contaminated duta-generating distribution
Data-driven model
Out-of-sample environment
P.
F
Given that the statistician knows that the data has been contaminated, a natural policy class to consider involves rectifying/corrrccingg the contamination, and, for this, we introduce rectification set R(F,.) which models set of possible pre- co-commimiaiionn distributions based on the knowledge of the empirical measure F To ensure good performance in terms of the optimal expected population approach naturally induces the Figure 1:
Decision Making Cycle
the use of alternative approaches. In the following paragraphs, we describe two of these approaches, DRO and robust statistics, by treating them as adversarial games. The crucial distinction lies in the timing of the contamination or attacks.
(i) (DRO: P P. P)) The attack occurs in the post-decision stage. In this scenario, the out- of-sample environment diverges from the data-generating distribution P., and no contamination occurs before the decision, implying that P. F'. For example, in adversarial deployment scenarios, malicious actors can deliberately manipulate the data distribution compromise the performance of trained models. With full access to our trained machine learning model, the adversary endeavors to create adversarial examples specifically designed to provoke errors in the model's predictions.
To ensure good performance in terms of the optimal expected population loss over the out-of-sample distribution, the DRO framework introduces an uncertainty set ((P,,) to encompass discrepancies between the in-sample-distributio I", and the out-of-sample distribution P. Subsequently, the DRO formulation minimizes the worst-case loss within this uncertainty set, thereby aiming to solve
min sup Eq[e(0, Z)].
(3)
DEO QEB(P,)
(ii) (Robust Statistics: IP P The contamination occurs in the pre-decision stage. Many real-world datasets exhibit outliers or measurement errors at various stages of data generation and collection. In such scenarios, the observed samples are generated by a contaminated distribution P'', which differs from the underlying uncontaminated distribution IP.. But, the outple distribution equals the original clean distribution. In contrast with DRO, the adversary corrupts the clean data according contamination model prior to training. Our objective is to clean the data during the training phase to achieve robust classifier. It is noteworthy that the attacker does not have access to the specific model to selected by the learner.
loss over the clean distribution, the rectificatio/deconaaminnt following min-min strategy:
min min Eq [e(0, Z))
DEB QER(P.)
Our goal in this paper is develop (4), which is completely distinct from DRO.
3 Automatic Outlier Rectification Mechanism
We now introduce our primary contribution by delineating (4). Our estimator is crafted to incorporate outlier rectification and parameter estimation within unified optimization framework, facilitating au- tomatic outlier rectification. A natural question arises from (4): how do we construct the rectification set in the space of probability distributions correct the observed data set? this paper, we employ an optimal transport distance to create a ball centered at the contaminated empirical distribution F
Definition 3.1 (Rectification Set). The optimal transport-based rectification set defined as R(P',) = (Q E P(2): (. < 8),
(5)
where 5 > 0 is a radius, and D. is specific optimal transport distance defined in Definition 3.2.
Definition 3.2. (Optimal Transport Distance [Peyré et al., 2019, Villani, 2009]) Suppose that c(;:)) :):
Z x Z + [0, oo] is lower semi-continuous cost function such that c(z, z²) 0 for all z' E Z
satisfying = zí. The optimal transport distance between two probability measures PP, Q E P(Z) is
D.(P, ( xEP(2xZ)
inf
Ex [x 2')] TM = P,, πz =

---

Here, P(2 x 2) is the set of joint probability distribution π of (Z, 2') supported on 2 x Z while Ty and कz. respectively refer the marginals of Z and Z' under the joint distribution x.
If we consider d(z=²) = ||= - ='1"" as a metric defined on R4, where r E [1, 00), then the distance metric / (IP Q) corresponds to the r-th order Wasserstein distance [Villani. 2009]. In this paper, we pioneer the utilization of concave cost functions, exemplified by c(z, = || z'1l" where E (0, 1) in statistical robust estimation. We note that D& (P, Q) (as opposed to D 1/rr (I,, Q) is also a metric for E [0. 1). The rationale behind selecting a concave cost function is intuitive: it promotes what we colloquially refer to long haul transportation plans, enabling outliers to be automatically moved significant distances back towards the central tendency of data distribution. This, in turn, facilitates automatic outlier rectification. Concave costs promote long hauls due to their characteristic of exhibiting decreasing marginal increases in transportation cost. In other words, if an adversary decides to transport a data point by || units, it becomes cheaper to continue transporting the same point by an additional e distance compared to moving another point from its initial location. We make further remarks about the connection between our estimator and prior work in Appendix B.
Illustrative Example. We provide empirical evidence showcasing the efficacy of the concave cost function simulated data in Figure 2 and 3. This example shows that our concave cost function is critical for moving the outliers (orange) properly to the bulk of the clean distribution (blue). For the concave cost, only the outliers are rectified (green), resulting in the proper line of best fit.
Rectified D stribution Corrupted)
Recifiid Distribution Corupttd)
Distribution
Compupted)
7.5
Grean
7.5-
Green Date
7.5- Glean
cud Duta
Dubist
Cudier Duta
Duttee Dats (Aacltee))
5.0
Duttoe bats [eeeifle)))
Sutler fres (BarlEed))
cround Tuth
croun) Tuth
arandd ruth
2.5
Concaue Ractilee
Concuue
0.0
0.0
0.0
-2.5
-2.5
-2.5
-5.0
-5.0
-5.0
(a) Budget 5
0
(b) Budget ō 0.9
(c) Budget 5 1.5
Figure 2: The rectified data generated by our estimator with concave cost function (r 0.5).
Rectified Distribttion
Rectified Distribation
Rectified Distribxtion
D, r (45% Corrupted)
6 0.9. 0.9.0 (= (45% Comuppte)
= fm5, (45% Corrupted)
7.5
Clean Data
Clean
7.5
Clean
out
out Dat
outic: Dura
3.0
5.0
5.0
2.5
Sectiter
Convex Gactiter
2.5
Convect factiter
2.5
2.5
2.5-
-5.0
4 -5.0
-5.0
0
-2
-2
2
(a) Budget 5 =
(b) Budget 8 = 0.9
(c) Budget 1.5
Figure 3: The rectified data generated by our estimator with convex cost function (r 2).
However, for the convex cost, regardless of the budget, all points both clean and corrupted are rectified towards each other instead, which results in an incorrect line of best fit. This illustrates succinctly the importance of our novel concave cost. More details on this example are given in Appendix F.1, and an additional example with non-clustered outliers is given in Appendix F.1.1.
4 Reformulation Results
With this intuition in hand, we focus on the derivation of equivalent reformulations for the infinite- dimensional optimization problem over probability measures in (4). These reformulations provide us with fresh perspective on adaptive quantile regression or estimation, where our introduction of an efficient transporter rectifies the empirical distribution, eliminating the influence of outliers.
To begin with, we can transform problem (4) into an equivalent finite-dimensional problem by leveraging the following strong duality theorem.
Proposition 1 (Strong Duality). Suppose that ((0,-) lower semicontinuous and integrable under F' for any 0 O. Then, the strong duality holds, i.c.
5

---

inf Bo(e(0: QER(E))
max Ep, A30
min ((0: + Ad(2, xc(2,Z)) - Ap.
LE22
The proof is essentially based on the strong duality results developed for Wasserstein distributionally robust optimization problems [Zhang et al., 2022, Li et al., 2022. Blanchet and Murthy, 2019, Gao and Kleywegt, 2022, Mohajerin Esfahani and Kuhn, 2018], which allows us to rewrite the original problem (4) as (PF Ee(e(0: 2)]
SuP Eg -((;: 2)].
We proceed to examine several representative examples to better understand the proposed statistically robust estimator (4). We begin with mean estimation to showcase
estimator's performance on one of the most classic problems of point estimation which can be easily understood. We then give an example for least absolute deviations (L.D) regression, one of the most important problems of robust statistics. LAD regression builds a conceptual foundation which leads into a discussion of our framework in more general cases and problem domains.
4.1 Mean Estimation
The mean estimation task is widely recognized as a fundamental problem in robust statistics, making it an essential example to consider. In this context, we define the loss function as {(0;=) 1 - =||- It is worth noting that when 8 O, Problem (4) is equivalent the median, which has already been proven effective in the existing literature. However, beyond the equivalence to the median, there are additional benefits to be explored regarding the proposed rectification set. By deriving the equivalent reformulation and analyzing the optimal rectified distribution, we can gain further insights into how the proposed statistically robust estimator operates. This analysis provides valuable intuition into the workings of the estimator and its advantages.
Theorem 2 (Mean Estimation). Suppose that Rd l(0:2) |10 - =1l and the cost function is defined as c(z, (') = I| - z||" where r E (0, 1). Without loss of generality, suppose that the following condition holds
110 211l &
|10 2/21l E < 110 2/11-
(6)
Then, we have the inner minimization of (4)
k(0)-
1 (
no= Li=k(0)+1 110 -2(11" 2/11" ) 110- =hoo)ll
max
||0
2111
(7)
110 <000) l1"
1=1
where k(0)= max (k: 1
E 110 =||l" > 0}.
kEju]
We give the proof in Appendix C.I.
Remark 4.1. The resulting reformulation problem can be viewed finding a quantile of 110 2'll controlled by the budget 6. If we have a sufficient budget 8 such that Ep, [||0 2''l'] < 6, it implies that all data points have been rectified the value of 0. Consequently, the minimum value of min.eR((P.)) [q Z)] will be equal to zero. In nontrivial cases, when given a budget 6 >> 0 and the current estimator our objective
identify and rectify the outliers in the observed data. To achieve this, we start by sorting the data points based on their loss value ||e z/ll- We relocate the data points starting with the one having the largest loss value, 2/x- The goal to move each data point towards the current mean estimation 0 until the entire budget is fully utilized.
Building upon the proof of Theorem 2, we can establish the following characterization of the rectified distribution. The concave cost function ||2 z'||" (with r E (0, 1)) plays a pivotal role in this context by endowing the perturbation with a distinctive long haul structure. Intuitively, for each data point, we can only observe two possible scenarios: either the perturbation is zero, indicating no movement or the data point is adjusted to eliminate the loss. In this process, the rectified data points are automatically identified as outliers and subsequently rectified.
Proposition 3 (Characterization of Rectified Distribution). Assuming the same conditions as stated in Theorem 2, we can conclude that the optimal distribution Q*
k(0)
Q' (dz)
1
52 (dz) + "
8 22
+ 11 k(0) + - 50 (d22
no E
110
where 1) =
Remark 4.2. The existence of optimal solutions follows directly from Yue et al. [2022, Theorem 2].

---

4.2 Least Absolute Deviation (LAD) Regression and More General Forms
Following the same technique, we can also derive the least absolute deviation case.
Theorem 4 (LAD Regression). Suppose that 2 == X' x y = R Rd+1 E(0, 2) = -ly - dxxl and the
cost function defined c(z, =²) ||z =||" where r E (0. 1) and || is the norm. Without
loss of generality, suppose that lly1 0T 2'1 < 11 0" x/2 & ...
& ll1h 0" 24, ll. we have the
inner minimization of (4)
- ll 0" r'll" + 1 no' -i 1111 8 xill")
Σ
Ilukcos 0' dxeepho0)
max
V12(00 8" x''x((n) ||"
where k(() max { - 11 - 0T > 8*) and 8' 60. k
We give the proof of Theorem 4 in Appendix C.I. The structure of the optimal rectified distribution also resembles that of Proposition 3: the rectified data points are shifted towards the hyperplane y 8 z that best fits the clean data points. For more detailed explanation of the long haul structure and extensions to more general cases, interested readers are encouraged to refer to Appendix B.
4.3 Computational Procedure
Our procedure is motivated by the empirical efficacy of subgradient descent for training deep neural networks, described in e.g. Li et al. [2020]. In each iteration, given the current estimate 0, compute the optimal rectified distribution. For simple applications such as mean estimation and least absolute deviation regression, we can solve this optimization problem efficiently using the quick-select algorithm or by utilizing an existing solver for linear programs to obtain the optimal solution for the dual variable A, which we denote X² Then, we employ the subgradient method on the rectified data, iterating until convergence. This optimization process automatically rectifies outliers using& fixed budget 8 across all iterations. As the value of l changes, the same budget 8 for all iterations results in varying the quantile used for selecting outliers. Thus, our estimator can be regarded as an iteratively adaptive quantile estimation approach.
We now concretely detail the procedure for LAD regression. The procedure for mean estimation is analogous and results from a simple change in the loss function. We must start by initially addressing the computation of the optimal dual variable X for the inner minimization over the probability space. Without loss of generality, we recall that this problem is
N11OT 41"" |
Σ min 110T 2111,
max 2
A5.
(8)
72
We show in the Appendix in Section C.I by Lemma 6 that this problem can be solved by applying the quick-select algorithm to find the optimal X* and the knot point k(O)) which demarcates estimated outliers. Applying this lemma yields estimation approach, which is described in Algorithm 1.
Algorithm 1: Statistically Robust Estimator
Data: Observed data {24) initial point 0, stepsizes a₁ >> O;
for t = T do
Sort the observed data { via the value 1|0// - 03 x/]l-
2
2. Quick-Select algorithm to get the knot point k(O,)) and the optimal Xt:
Subgradient step on the detected clean data:
k(0,)
0.+1 02
O0
sgn 0 x vl) x.
n
s end
Additional details on an alternative lincar program approach for estimating k(0,) and X are given in Appendix D. We also propose a procedure for estimating more general regression models in Appendix D.2. Further details on the optimization problem are given in Appendix D.3.
Limitations. Appendices D.2 and D.3 contain explanations of the limitations of our approach, which include the difficulty of solving the associated optimization problems and the nature of the derivation of our optimization algorithm for more general regression models.
7

---

Experimental Results
In this section, we demonstrate the effectiveness of the proposed statistically robust estimator through various tasks: mean estimation, LAD regression, and two applications to volatility surface modeling.
5.1 Mean Estimation and LAD Regression Experiments
We perform simulation experiments with mean estimation and least absolute deviation linear regres- sion for our estimator to illustrate its efficacy. For mean estimation, we compare our estimator with the mean, median, and trimmed mean estimators. We observe that our estimator outperforms all other methods, despite providing the oracle corruption level to the trimmed mean estimator. Our results are displayed in Table below.
Table 1:
We compared our estimator with several standard mean estimation methods by evaluating the average loss on clean data points across various corruption levels. our evaluation, we set the percent level of trimmed mean equal to the unknown ground truth corruption level. The hyperparameters for our estimator, namely
0.5 and r = 0.5, remained constant across all corrupted levels. The last row in the comparison table represents the percentage of all points rectified by our method. Error bars represent two standard deviation confidence intervals assuming normal errors.
Corruption Levels
20%
30%
40%
45%
49%
Mean
5,009 1 0.056
7.509 + 0.080
10.007 1 0.098
11.248 0.098
12.257 1 0.114
Median
1.680 1: 0.114
1.831 1: 0.118
2.286 0.192
2.843 1 0.208
4.203 0.344
Trimmed Mean
1.739 1 00108
1.947 +: 0.14
2,456 1 0.212
3,047 1 0.174
4.399 i 0.360
Ours
1.620 1 0.106
1.700 1 0.106
1.957 1 0.160
2.251 1: 0.150
2.899 1 0.240
Ours, Reetified
10.03%
10.12%
10.24%
10.35%
10.55%
For LAD regression, we compare estimator with the OLS, LAD, and Huber regression estimators. We find that, again, our estimator outperforms all other methods. A complete overview of these two experiments can be found in Appendix
5.2 Options Volatility Experiments
In this section, we conduct empirical studies on real-world applications in fitting and predicting option implied volatility surfaces. Options are financial instruments allowing buyers the right to buy (call options) or sell (put options) an asset at a predetermined price (strike price) by specified expiration date. In this study, we focus on European-style options, which can only be exercised at expiration. Option prices are influenced by the volatility of the underlying asset's 's price. Implied volatility (IV), derived from an option's 's market price, indicates the market's 'ss volatility expectations. The implied volatility surface (IVS) represents the variation of IV across different strike prices and times to maturity. Accurate IVS modeling is crucial for risk assessment, hedging, pricing, and trading decisions. However, outliers in IV can distort the IVS, necessitating robust estimation methods. Our approach addresses this by estimating the IVS in the presence of outliers. We conduct two experiments to demonstrate the effectiveness of our statistically robust estimator: (a) using kernelized IVS estimator and (b) using a state-of-the-at deep learning IVS estimator.
5.2.1 Kernelized Volatility Surface Estimation
Data. Our data set comprises nearly 2,000 option chains containing daily US stock option prices from 2019-20211 The options data is sourced from WRDS, a financial research service available to universities. The option chains were identified containing significant outliers by our industry partner, a firm providing global financial data and analysis. We were blinded to this choice. We randomly draw training set and test set from each chain and estimate surfaces. We assess the surfaces' out-of-sample performance using mean absolute percentage error (MAPE) and the discretized surface gradient (defined as V.).. MAPE evaluates the error of the surface versus observed implied volatilities. V evaluates the smoothness of the surface. Further details can be found in Appendix G.2.
Benchmarks. We compare our estimator developed in Theorem 4 to two benchmarks. The first kernel smoothing (denoted "KS"), a well-established method for estimating the IVS [Ait-Sahalia and Lo, 1998]. In KS, each point on the IVS is constructed by weighted local average of implied volatilities across nearby expiration dates, strike prices, and call/put type. The weights are given by a kernel matrix which depends these three features of an option (see next paragraph). The second
8

---

benchmark is two-step "remove and fit" kernel smoothing method (denoted "2SKS") which first attempts to remove outliers via Tukey fences [Hoaglin et al., 1986] before applying the KS method.
Kernelized Regression Problem. We now describe the kernel and our estimator. Following the convention in Ait-Sahalia and Lo [1998], the ith option in chain is featurized as the vector X B x (log Tv, uf (A),, Iea)())). where Ti is the number of days to the option's 's expiration date, A, E [-1,1] is relative measure of price termed the Black-Scholes delta, u(ar)
=la30 + (1 + => and Tealic)) is if the option is a call option and O if the option is a put option. The goal is to fit the pair (yf: the implied volatility of option i, z/; the features) via a kernelized regression model. We choose Gaussian-like like kernel K2, (zr, x") to measure the distance between options and x' as (z,z, x²) = cxp(- (-1(x 2') 22)) where division of x x' by h is element-wise for a vector of bandwidths f E R When the budget 8 = O, we want to solve min Σi=1 1a u/ly/ 0% where 0; is the i-th entry of a vector of vegas for { and Kj, is the i-th row of the kernel matrix. The implied volatilities are weighted by vega U/ to improve surface stability as in Mayhew [1995]. We conducted a comparison between our estimator and the benchmarks. We note that the KS method can be regarded as a standard kernelized least square approach [Hansen, 2022].
Results. Our approach improves upon the benchmark approach in both MAPE and VŠ on the out-of-sample test sets, showcasing the importance of jointly estimating the rectified distribution and model parameters with our estimator. The results of our experiment are displayed in Table
1. We compare the KS and 2SKS benchmarks against our estimator with both fixed 8 = 0.01 (chosen to correspond to a 1% change in volatility) and 6 chosen by cross-validation. More details can be found in Appendix G.2. Our estimator outperforms others with a mean MAPE of 0.225, compared to 0.294 for KS and 0.236 for 2SKS, showing 23% and 5% improvement respectively. For surface smoothness, our estimator achieves a mean DSG of 6.5, significantly better than 20.2 for KS and 7.5 for 2SKS, indicating 67% and 13% improvements. Notably, our industry partner applied our estimator to an unseen test set collected after this paper was written and was able to release to production 25% more surfaces than they had before when using their existing proprietary method.
Table 2: Results of our experiment with kernelized IVS estimation.
Model MAPE
0.5% Quantile
5% Quantile
Median
Mean
95% Quantile 99.5% Quantile
KS 2SKS
0.026
0.068
0,232
0.294
0.677
1.438
0.026
0.056
0.172
0.236
0.602
1.389
Ours (8 = 10-2)
0.028
0.057
0.170
0.225
0.535
1.207
Ours (CV)
0.028
0.057
0.169
0,224
0.534
1.240
Model V
0.5% Quantile
5% Quantile
Median
Mean
95% Quantile
99.5% Quantile
KS 2SKS
0.313
1.606
14,434
20.188
57.797
108.901
0,050
0.124
2.229
7.491
33,276
78.144
Ours (8 10-2)
0.048
0.121
1.898
6.502
28.064
66.410
Ours (CV)
0.043
0.109
1.513
5.079
21.096
55.005
Remark 5.1. To contextualize these results, we note that the difference in MAPE of a surface and an option chain containing outliers and the MAPE of a surface and an option chain containing no outliers will not be large if the set of outliers is small. Consider the MAPE of the same surface for two different option chains of size n. Oj and Oz, where small fraction k/n n of the options of O2 are outliers. Supposing the modal APE 0.3 and the outlier APE is a considerable 1.0, for an options chain with n = 50 and just l = 5, the MAPE difference will be only k+0.3(n-k) 0.3 = 0.07.
We perform an additional experiment with the same dataset in Appendix G.I which demonstrates the usefulness of our method for estimating an IVS for use on the trading day after the initial contaminated surface is observed. We find similar outperformance of our method versus the benchmark methods.
5.2.2 Deep Learning Volatility Surface Estimation
In this section, we apply our statistically robust estimator to tfte-offthhe-ar deep learning prediction approaches for modeling volatility developed Chataigner et al. [2020]. In this work, deep networks are used to estimate local volatility surfaces. The local volatility or implied volatility function model introduced by Rubinstein [1994], Dupire et al. [1994], and Derman et al. [1996] is a surface which allows for as close of a fit as possible to market-observed implied volatilies without arbitrage, which is of interest to many market participants (smoothing methods, in contrast, do not make this guarantee).
Further background and details for this section are available in Appendix G.3.

---

We select the data set from Chataigner et al. [2020] consisting of (options chain, surface) pairs from the German DAX index. We first contaminate the chains by replacing an e fraction of each price with 10p. We then test our estimated surface against the true surface. To estimate price and volatility surfaces under data corruption, we applied our statistically robust estimator to the benchmark approach. We use the Dupire neural network of Chataigner et al. [2020] as benchmark, which estimates the surface under local volatility model assumptions and no-arbitrage constraints, This method enforces these conditions on the surface using hard, soft, and bybrid hard/soft constraints.
We evaluate our robust estimator using the same metries as Chataigner et al. [2020], RMSE and MAPE, repeated over three trials with different random seeds. Our approach outperforms the baseline approach across all averages, and does so more clearly as the corruption level increases, despite the strong regularizing effect of the no-arbitrage constraints and enforcement of Dupire's 's formula. Our improvement most impactful for the most accurate model utilizing soft arbitrage constraints. For this model, the test set RMSE and MAPE are reduced by 33% and 34%. This experiment displays the efficacy of our estimator in state-of-the-an deep learning approach to volatility modeling.
Table 3: Results of our experiment with deep learning surface estimation.
Panel Results Constraint Type
Model
Hard Constraints
Hybrid Constraints
Soft Constraints
Dupire NN RMSE
0.125
0.140
0.044
Our RMSE
0.110
0.131
0.029
Dupire NN MAPE
0.343
0.546
0.111
Our MAPE
0.310
0,508
0.074
Panel B: Results by Corruption Level
Corruption Level
= 20%
e == 30%
8 40%
Dupire NN RMSE
0.077
0.091
0.168
Our RMSE
0.075
0.084
0.141
Dupire NN MAPE
0.061
0.070
0.115
Our MAPE
0.060
0.066
0.097
6 Conclusion
In conclusion, we propose an automatic outlier rectification mechanism that integrates outlier cor- rection and estimation within a unified optimization framework. Our novel approach leverages the optimal transport distance with concave cost function to construct a rectification set within the realm of probability distributions. Within this set, we identify the optimal distribution for conducting the estimation task. Notably, the concave cost function's "long hauls" attribute facilitates moving only a fraction of the data distant positions while preserving the remaining dataset, enabling efficient outlier correction during the optimization process. Through comprehensive simulation and empirical analyses involving mean estimation, least absolute regression, and fitting option implied volatility sur- faces, we substantiate the effectiveness and superiority of our method over conventional approaches. This demonstrates the potential of our framework to significantly enhance outlier detection integrated within the estimation process across diverse analytical scenarios.
Acknowledgments and Disclosure of Funding
J. Blanchet, M. Pelger, and G. Zanotti acknowledge support from Morgan Stanley Capital Interna- tional. Support was provided to J. Li by the Air Force Office of Scientific Research under award number FA9550-20-1-0397,
References
Yacine Ait-Sahalia and Andrew W Lo. Nonparametric estimation of state-price densities implicit in financial asset prices. The Journal of Finance, 53(2):499 547, 1998.
Harbir Antil, Sean P Carney, Hugo Diaz, and Johannes O Royset. Rockafellian relaxation for pde-constrained optimization with distributional uncertainty. arXiv preprint 4rXiv:2405.00/77 2024.
10

---

Güzin Bayraksan and David K Love. Data-driven stochastic programming using phi- thii-ivergences. In The operations research revolution, pages 1-19. INFORMS, 2015.
Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Management Science, 59(2):341-357, 2013.
Jose Blanchet and Yang Kang. Distributionally robust groupwise regularization estimator. In Asian Conference on Machine Learning, pages 97-112. PMLR, 2017.
Jose Blanchet and Karthyek Murthy. Quantifying distributional model risk via optimal transport. Mathematics of Operations Research, 44(2):565-606 600, 2019.
Jose Blanchet, Jiajin Li, Sirui Lin, and Xuhui Zhang. Distributionally robust optimization and robust statistics, arXiv preprint arXiv:2401.14665, 2024.
George EP Box. Non-normality and tests on variances. Biometrika, 40(3/4):318-33) 1953.
Marc Chataigner, Stéphane Crépey, and Matthew Dixon. Deep local volatility. Risks, 8(3):82, 2020.
Louis L Chen and Johannes O Royset. Rockafellian relaxation in optimization under uncertainty: Asymptotically exact formulations. arXiv preprint arXiv:2204.04762, 2022.
Tat-Jun Chin, Jin Yu, and David Suter. Accelerated hypothesis generation for multistructure data via preference analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4): 625-638, 2011.
Stéphane Crépey. Calibration of the local volatility in trinomial tree using tikhonov regularization. Inverse Problems, 19(1):91, 2002.
Stéphane Crépey. Delta-hedging vega risk? Quantitative Finance, 4(5):559-579, 2004.
Damek Davis, Dmitriy Drusvyatskiy, Sham Kakade, and Jason D Lee. Stochastic subgradient method converges on tame functions. Foundations of computational mathematics, 20(1): 20(1):119-154, 19-154, 2020.
Erick Delage and Yinyu Ye. Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations research, 58(3):595-612, 2010.
Emanuel Derman, Iraj Kani, and Joseph ZoZou. The local volatility surface: Unlocking the information in index option prices. Financial Analysts Journal, 52(4):25-366 1996.
David L. Donoho and Richard Liu. The automatic robustness of minimum distance functionals. The Annals of Statistics, 16(2):552-586. 1988a.
David L Donoho and Richard C Liu. Pathologies of some minimum distance estimators. The Annals of Statistics, pages 587-608, 1988b.
Bruno Dupire et al. Pricing with smile. Risk, 7(1):18-22 20, 1994.
Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications image analysis and automated cartography. Communications of the ACM, 24 (6):381-395, 1981.
Rui Gao and Anton Kleywegt. Distributionally robust stochastic optimization with wasserstein distance, Mathematics of Operations Research, 2022.
Joseph Diez Gergonne. Dissertation sur la recherche du milieu le plus probable. Etc. Annales math. pures appl, 12(6):181-204 1821.
Frank R Hampel. Contributions to the theory of robust estimation. University California, Berkeley, 1968.
Frank Hampel. A general qualitative definition of robustness. The annals of mathematical statistics, 1887-1988, 1971.

---

Bruce Hansen. Econometrics, chapter 19: Nonparametric Regression. Princeton University Press, 2022.
Fumio Hayashi. Econometrics. Princeton University Press, 2000.
Xuming He and Stephen Portnoy. Reweighted Is estimators converge the same rate as the initial estimator. The Annals of Statistics, pages 2161-2167, 1992.
David C Hoaglin, Boris Iglewicz, and John W Tukey. Performance of some resistant rules for outlier labeling. Journal of the American Statistical Association, 81(396):991-99 999, 1986.
Peter Huber. Robust estimation of a location parameter. Ann Math Stat, 35:73-101, 1964.
John Hull and Sankarshan Basu. 27.3: The IVF Model, page 659-660. Pearson, 11 edition, 2022.
María Jaenada, Pedro Miranda, and Leandro Pardo. Robust test statistics based restricted minimum rényi's pseudodistance estimators. Entropy, 24(5):616, 2022.
Nan Jiang and Weijun Xie. Distributionally favorable optimization: A framework for data-driven decision-making with endogenous outliers. SIAM Journal Optimization, (4(11):199455) 2024.
Sumin Kang and Manish Bansal. Distributionally risk-receptive and risk-averse network interdiction problems with general ambiguity set. Networks, (1)):33-2, 2023.
Daniel Kuhn, Peyman Mohajerin Esfahani, Viet Anh Nguyen, and Soroosh Shafieezadeh-Abate Abadeh.. Wasserstein distributionally robust optimization: Theory and applications in machine learning. In Operations research & management science the age of analytics, pages 130-166. Informs, 2019.
Erich Lehmann and George Casella. Theory of Point Estimation. Springer Science & Business Media, 2006.
Jiajin Li. Anthony Man-Cho So, and Wing- Win Ma. Understanding notions of stationarity in nonsmooth optimization: A guided tour of various constructions of subdifferential for nonsmooth functions. IEEE Signal Processing Magazine, 37(5):18-31, 2020.
Jiajin Li, Sirui Lin, Jose Blanchet, and Viet Anh Nguyen. Tikhonov regularization is optimal transport robust under martingale constraints. Advances in Neural Information Processing Systems, 2022.
Stewart Mayhew. Implied volatility. Financial Analysts Journal, 31(4):8-20, 1995.
Warwick Millar, Robust estimation via minimum distance methods. Zeitschrift für Wahrschein- lichkeitstheorie und verwandte Gebiete, 55(1):73-89 1981.
Peyman Mohajerin Esfahani and Daniel Kuhn. Data-driven distributionally robust optimization using the wasserstein metric: Performance guarantees and tractable reformulations, Mathematical Programminn, 171(1):115-16 2018.
Sloan Nietert, Ziv Goldfeld, and Soroosh Shafiee. Outlier-robust wasserstein DRO. Advances in Neural Information Processing Systems, 36, 2023,
Chanscok Park, Ayanendranath Basu, and Srabashi Basu. Robust minimum distance inference based combined distances. Communications in Statistics-Simulle and Computation, 24(3):653-673, 1995.
William C Parr and William R Schucany. Minimum distance and robust estimation. Journal of the American Staristical Association. 75(371):616-6) 6-62, 1980.
Benjamin Peirce. Criterion for the rejection of doubtful observations. The Astronomical Journal, 2: 161-163, 1852.
Gabriel Peyré, Marco Cuturi, et Computational optimal transport: With applications data science. Foundations and Trends@ in Machine Learning, (1(-66)355-607,22 2019,,
Peter J. Rousseeuw and Annick M. Leroy. Robust regression and outlier detection. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. John Wiley Sons, Inc., 1987.
12

---

Mark Rubinstein. Implied binomial trees. The Journal of Finance, 49(3):771-88)) 1-88 1994.
Soroosh Shafieezadeh Abadeh, Peyman M Mohajerin Esfahani, and Daniel Kuhn. Distributionally robust logistic regression. In Advances in Neural Information Processing Systems 28, 2015.
Soroosh Shaficezadch-Abatee Daniel Kuhn, and Peyman Mohajerin Esfahani. Regularization via mass transportation. Journal of Machine Learning Research, 20(103):1-68, 2019.
Aman Sinha, Hongseok Namkoong, and John Duchi. Certifying some distributional robustness with principled adversarial training. In International Conference on Learning Representations, 2018.
Robin Thompson. A note on restricted maximum likelihood estimation with an alternative outlier model. Journal of the Royal Statistical Society: Series (Methodological), 47(1):53-55, 1985.
John W Tukey, A survey sampling from contaminated distributions. Contributions to probability and statistics, pages 448-485, 1960.
John W Tukey, Exploratory Data Analysis. Addison-Wesley, 1977.
Aad W Van der Vaart. Asymptotic Statistics, volume 3. Cambridge University Press, 2000.
Cédric Villani. Optimal Transport: Old and New, volume 338. Springer, 2009.
Wolfram Wiesemann, Daniel Kuhn, and Melvyn Sim. Distributionally robust convex optimization. Operations Research, 62(6):1358-13) 2014.
Man-Chung Yue, Daniel Kuhn, and Wolfram Wiesemann. On linear optimization over wasserstein balls. Mathematical Programming. 195(1-2):1177-1122, 2022.
Luhao Zhang, Jincheng Yang, and Rui Gao. A simple and general duality proof for wasserstein distributionally robust optimization. arXiv preprint arxiv:2205.00362, 2022.
Banghua Zhu, Jiantao Jiao, and Jacob Steinhardt. Generalized resilience and robust statisties. The
Annals of Statistics, 20(4))2256-2283, 2022.
13

---

Appendix
A Organization of the Appendix
We organize the appendix as follows:
The organization of the Appendix is given in this section (Section A).
Additional details the connection between estimator and prior work given in Section B.1.
Additional explanation of the long haul structure induced by our estimator and extensions to more general forms of problems given in Appendix B.2.
The proof of Theorem 2 and Theorem 4 are given in Section C.I.
The long haul structure for linear regression with a concave cost function is provided in Section C.2.
The detailed computational procedure for mean estimation, linear absolute regression, and more general cases can be found in Section D.
The main experiments for mean estimation and LAD regression are described in Appendix E.1 and E.2 respectively. Additional details are given in the following appendices.
Additional details on the illustrative example in the text is given in Section F.1.
Additional details on mean estimation simulations are given in Section F.2.
Additional plots for the mean estimation simulations for the concave cost function are given in Section F.3.
Additional plots for the mean estimation simulations for the convex cost function are given in Section F.4.
Additional details on LAD regression simulations are given in Section F.5.
Additional plots for the LAD regression simulations for the concave cost function are given in Section F.6.
Additional plots for the LAD regression simulations for the convex cost function given in Section F.7.
Additional option volatility surface experiment details are given in Section G.I.
Additional information on the volatility surface data set and losses are given in Section G.2. Background and additional details the deep learning volatility modeling experiment given in Section G.3.
14

---

B Estimator Remarks
B.1 Connections Prior Work
Additional details on connections to prior work are given in the following remark:
Remark B.1. (i) When we disregard the outer minimization concerning estimation parameters, the inner minimization problem over probability measures is related to the minimum distance functionals-based estimator, initially proposed by Donoho and Liu [1988a]. The estimator is obtained by first projecting the corrupted distribution Pº, onto a family of distributions G under a certain distribution discrepancy measure D. Then, the optimal parameters distribution. However, even with additional information about our contamination models, this "two-stage" procedure has two major drawbacks: the difficulty in choosing appropriate family of distributions G and the inherent computational challenge to project probability distributions. Moreover, the first stage (projection) is usually sensitive to the choice of family G. In contrast, propose a novel approach that integrates outlier rectification (i.e., explicit projection) and parameter estimation into a joint optimization framework in (3). (ii) This min-min strategy has also been explored in Jiang and Xie [2024] using an artificially constructed rectification set. Their primary focus is to utilize the min-min formulation to recover well-studied robust statístics estimators. In contrast, our focus is on a new conceptual framework for formulating novel estimators which have not been studied before.
selected for the resulting
B.2 Long Haul Structure and Extensions
An explanation of the long haul structure and extensions to more general forms are given in this subsection.
The proposed rectification set has the potential for broader applicability across various applications and problem domains. This versatility makes the rectification set valuable tool for addressing and improving solutions in diverse problem settings in the future.
(i) (Concave cost function) From Proposition 1, the optimal rectification admits Abest E arg mingez {(0, 2') + Ac(z, Suppose we select a concave cost function e that grows strictly slower than the loss function f. In this scenario, when the budget is small, the rectified data point Abest consistently exhibits the long haul structure, ensuring automatic outier identification properties. This implies that the rectification process effectively identifies outliers, as the influence of the cost function dominates over the loss function for small budgets. To further illustrate the concept, we can consider linear regression problem with the squared loss function E(0, 2) (y 0 x:)² or nonlinear loss using the r-th norm as the cost function. We refer the readers to Appendix C.2.
(ii) (Other distribution metrie/aliscrppancy) Based on the two examples discussed earlier, we can infer that the optimal transport distance-based rectification is particularly suitable for regression or problems with continuous response variables. This is because the budget 8 is used to compensate for the loss caused by identified outliers. However, in classification tasks, where the loss function may be less sensitive, the effectiveness of the optimal transport distance-based approach may be limited. To address this limitation and complement the rectification set for classification tasks, an alternative approach is to use di-divergences. This approach has already been proposed in a study by Chen and Royset [2022], Antil al. [2024] for handling outliers image classification tasks. The d-divergence-basee rectification set operates by adjusting the weight assigned to outliers, effectively minimizing their impact on the overall data set. By reducing the weight placed on the detected outliers, the rectification set aims to remove their influence from the data set. This selective adjustment allows for the identification and removal of outliers, leading to a more refined and reliable data set.
15

---

C Proof Details
C.1 Proof of Theorem 2 and Theorem 4.
To start with, we give two crucial lemmas and 6.
Lemma 5. Suppose that a, b, 0 and (0, 1), have
Aa" a, br
min bæ Ax min # 0,000]
Proof. The argument is easy. The function g(z) br Ax" is concave the second derivative always negative:
g() Ar(r - 1)x"=2
x > 0.
Lemma 6. Suppose that there is increasing sequence 0 E x1 < x2 < Fn. The optimal solution of
max X
a min f2,, Axn} A8
X" o/ 1--r where
k max kE(m]
k:
au > 8
i=k
Here a; E 10,1), a; = 1, > 0 and r (0, Moreover, the optimal function value is
1 (5 2i= (.0))
max
ag.TA 0).
Proof. Without loss of generality, we assume that 0 < x < <22 < ... < di as the zero part of the sequence does not affect the result.
First, given a fixed IR+ and the inequality I++1 < Ao our goal is to show that Ie < AxT. Let It 7 where 7) (0, 1]. Then, we have
X 17"
17441 mdxf41
Considering fixed E R.,, we can express the objective function as:
k-1
Σ a4 min Ar - X5
Σ o, + or min Aa +
Σ
a A8
imk+1
where 1 & k & and a weights associated with each x;. Two cases arise:
1. For all <t - ai AzZ, or for all ettnnaa ArT.
2. At the k-th point, we have 24 Ar
Since we are dealing with a concave piecewise linear function, the optimal À will be knot point (case 2). Otherwise, modifying X would increase the objective value. Therefore, we can establish the first-order optimality condition as follows:
Σ
axoi + paxoz = 8.
where & and [0, are determined through search using the quick-select algorithm. Conse- quently, the optimal solution is obtained as X
x 1-r and
m aaay > 8
k max
Σ
kE[n] ml
i=k
16

---

(i-Ei-i1 amd?) an
Thus, we further get μ
and the optimal function value admits
0447 + ag min A + A
Σ
a - Ad
1=++1
0:44 + ag min(2k, Aaoe Aua
04 + (1 - 11)00
04:24 + (1- (1- (6- 2x h+ actt))
n
Based on our discussion, we exclude the corner case where 1. Therefore, 5 is sufficiently large such that Σ a 8. It is evident that in this trivial scenario, the optimal function value is zero.
conclude our proof.
We now give the proof of Theorem 2:
Proof. Before prove the theorem, is worth highlighting that for any fixed 0, we can always sort { based on the error ||0 z{| to satisfy condition (6).
By the strong duality result in Proposition have
min QEB(E%)
Boll - z'1l]
max Ep,
min ||0 Z' All + || A8 LAER
A
max Ep, A
min |og||A//<|000>/
110 z ||||| + \|AI1" A8
max Epr [min {|[0 7111, A110 7'13 A6
A30
max
min A/1 A10 - +111²} - A8
X n
Tml
k(0)-1
100 zi11 + (1 no Σ ki=k(0)+1 110 2/11"
max
110
=1,0)
0
20,0 l1"
where the third equality follows from Lemma in Appendix C.I and the fifth one is due to Lemma in Appendix C.I.
The proof in the LAD regression case follows:
Proof. The proof follows a similar idea to that of Theorem 2. For any fixed 0, we can always sort {2 T= based on the error [ly/ 110= 2; to satisfy the condition. For simplicity, denote Ø = (0. 1)
17

---

and 2 = (z, v). By the strong duality result in Proposition 1, we have
min QER(P,)
BallY 0=xx] xI]
max Ep, A
min AgR4+1
100 Z' + 0 A + A|AI" A6
max Epr
min
110 z'1l
|01
*|
A8
A
2' Tdll.
A110 Z' |
(6)
max Ep, A
min
110T Z'Il,
A0
|101
λl 24I""
= max
min
|10"=411
X6
A n.
||
t=1
4(0)-1
D 2(0+(0)+1 110 =All"
1
10
2411 +
110T E
11,
max
| 1000 1|
where the third equality follows from the Holder inequality and || ||+ is the dual norm of || fourth one follows from Lemma and the sixth one is due Lemma 6.
||: the
This completes our proof.
L
C.2 Linear Regression with Concave Cost Function
In this subsection, aim illustrate example mentioned in Section 4.2. Suppose choose concave cost function c that grows strictly slower than the loss function f. In such scenario, when the budget is limited, the rectified data point Phess consistently demonstrates the long haul structure, thereby ensuring automatic outlier identification properties.
Example C.1. Suppose that Z Rd+1, E(0, 2) |y 0²'z1|² and the cost function is defined as =(zzz)= = || ==|" where r = 3 We want to study the best rectification distribution of
min Eg Elllly 8T 3/ QER(P())
As shall when the dual variable À is relatively small, the best rectification distribution will keep the long haul structure although the inner minimization problem is no longer concave.
For simplicity, we denote Ā = (0, 11 and = = (ær, y). By the strong duality result Proposition 1, we only have to focus on the inner minimization problem:
(00 + \|AI"
min AERd-1
min {(10-''' ||01- - A|||I"
|A|| 2T
Tell.
Next, we aim at clarifying the structured information of the following one-dimensional optimization problem:
min HAlE o. 19729) 211 1-11-
K(JAI) = (107 z' |01- A|//
In general, unlike the case of absolute loss, the function K(-) will not be concave. However, the optimal solution of the resulting optimization problem will active at the boundary when ă small and the optimal value of À is sufficiently large. Initially, let's overlook the constraint and express the
18

---

first-order optimality condition.
10 2 || 0
ا|
| 10" Z' 1101|
4
X
+110
10 z' 1'11011 |
4
By changing the variable = ||||| $ we have
30 + - ~0 1'11 O.
Now, we observe that g |0 + 10T 01. and our objective is find the positive root
of this cubic equation in one dimension. Also, the first derivative is Vg((5) = |||0||
10
Z'|l
01
The stationary point is 3
| and then the corresponding All
|
| z'
E
[0,
10"x1" z'
}.
3|
|0||
3
10 z'|
( z'
A
9(8*)
(v
11012
1 Z 110111
3|1011
31101, 4
3
(i.e., is sufficiently small), we have o(8*) > 0. As such, the critical point of the unconstrained optimization problem is not in the interval [0,
When À > 4 1 z'| 011_
0
3|10
3|ll1|_
|| 102211, In other words, we can conclude the optimal solution will be 0 and the solution of vanilla least square is already optimal.
2. When g(8*) O, know there are two solutions 8_ for g 0 where 3_ < 8 We know [I] will be increasing between [0, [ and decreasing between [ Thus, the optimal solution will be either 0 or B3, and ensures the long haul transportation structure. Different from the absolute loss, the cost function || =²|11/2 is not powerful enough move any points that achieve perfect fit the current hyperplane.
1.A.
1,A=1
1.15
1.1
Pinezion Valne
Flnciio Volue
1.05
0.96
0.2
0.4
0.5
0.8
0.2
0.4
0.5
0.8
[[A]
[[]]
(a) Case
(b) Case 2
Figure 4: Visualization
19

---

D Computational Procedure
D.1 Alternative Estimation Procedure for Inner Problem
We make the following remark on an alternative procedure for estimating k(Or) and À in the inner problem for mean estimation and LAD regression in Algorithm 1:
Remark D.1 (Alternative procedure for k(O,) and X*). It is illuminating to note that steps and 2 in Algorithm can be alternatively replaced with steps that solve a specific linear program for the knot point k0t)) and optimal X" Instead of applying the sorting and quick-select algorithm implied by Lemma 6, we can instead reformulate Equation (8) as a linear programming problem, i.e.,
max X20,tER"
D 4 A5
s.t.
t &
21l Vi [n] | & A|| =II', Vi E [n].
The problem in (9) then be solved instead of performing steps and 2 of Algorithm 1 to find the knot point k(0,) and the optimal X"
D.2 General Computational Procedure
We also propose a procedure for fitting more general regression models with our statistically robust estimator, which based on the approach for LAD regression above.
Although the procedure can be applied to any model which can be estimated via subgradient methods, focus deep learning models. We start by considering neural network fo parameterized by weights 0, and a loss function E(0, =) E(y, [ar),, where l is a gradient Lipschitz function with a would like solve the inner problem
constant T.. By Proposition
max Ep, min E(0;=) + Xc(2, Z') Ao. A
(10)
LEEZ
Although we do not prove reformulation result for this problem, solve the problem empirically by employing a computational procedure which is analogous to that of LAD regression. The procedure heuristic and does not enjoy the same guarantees as Algorithm 1 for LAD regression, but in experiments (see Section 5.2.2), we find that the performance is similarly robust to outliers and significantly outperforms benchmark models trained under empirical risk minimization. Our computational procedure follows in Algorithm 2.
Algorithm 2: Statistically Robust Optimization Procedure
Data: Observed data {2 ") initial point g()), stepsizes (1) >> 0, sampling distribution P'; batch size m < n.
for i 0...., do
1. Sample m points {=/)m² ~ IP,,
2. Sort the observed data {2() via the value ((0(t),2))
Quick-Select algorithm to get the knot point k(((t)) and the optimal X*
Subgradient step on the detected clean data:
k(((t))
g((+1) = g(t) - !0)
Σ
2t v*(0
where 0' (0(((), 24) 0ge(0()) 2))
2m1
end
Algorithm 2 is similar to Algorithm 1, but has two principal differences: (1) instead of optimizing over the entire data set, we optimize over mini-batches of size m which are drawn from data set of size via a sampling distribution PP, and (2) instead of taking the subgradient with respect to for the LAD regression problem in calculating the direction of descent, we instead take local subgradient with respect to 0 around each point z/,, each of which is contained within the subdifferetial On ) z(). The subgradient is in practice computed by the automatic differentiation capability of software package such as Pytorch, Tensorflow, or JAX. The budget parameter which controls the quantile which identifies outliers, is tuned via cross-vale coossvalidatiin.
20

---

Inregulaar Point
0.8
Objective Value
0.6
0.4
0.3
0.1
-0.5
1.5
Figure 5: Irregular objective function.
D.3 Optimization Details
Solving the optimization problems the prior sections can be complicated, as in general. the minimization of class of convex functions often leads to non-convex problems.
This issue becomes even more critical when dealing with our estimation problem, as these problems may lack weak convexity subdifferental regularity properties. One significant challenge in such problems the non-coincidence of different subdifferential concepts, such as Clarke, Fréchet, and limiting subdifferentials. Moreover, some calculus rules (summation and chain rules) do not always hold for these concepts. Thus, computing a finst-order oracle, which provides necessary information for optimization algorithms, be difficult in practice.
To concretely illustrate this challenge, we note that, intuitively, the graph of subdifferentially regular function cannot exhibit 'downward-facing cusps" [Li et al., 2020, Davis et al., 2020]. Here, we give an example of when the objective function (7) not regular, which may occur in general.
Example D.1 (Irregular Case (Three Point Masses)). Let IV _ + _ + J and 5 0.7. Based on the reformulation result given in Theorem 2, we can plot the curve of the loss function from our proposed estimator, see Figure 5.
From Figure 5, it is evident that there are at least two regions with "downward-facing cusps" (marked as irregularities in the figures), which could potentially pose computational challenges. Similar challenges arise in the training of deep neural networks, demonstrated in Figure 4 of al., 2020], particularly for two-layer neural networks. However, the "subgradient" method remains effective for addressing these challenges empirically. In our empirical investigation, we have observed similar phenomenon for our estimator.
21

---

E Mean Estimation and LAD Regression Experiments
In this section, we outline the auxiliary results of the mean estimation experiments and the primary results of the LAD regression experiments. Additional details are deferred to the following appendix section (Appendix F) for the convenience of readers.
E.1 Mean Estimation
We generate the corrupted data by combining two Gaussian distributions: (1 corruption level) x (0 2) and (corruption level) x (255, 2). For each corruption level in Table 1, 100 random trials performed for all results presented. We compare our estimator with several widely used baseline methods. Interestingly, we observe that even when setting hyperparameters as constants across all corruption levels, our estimator consistently outperforms the others. Notably, the trimmed mean, which incorporates the ground-truth corruption level, performs even worse than our estimator. Our estimator's 's outperformance is notable, as in this table we report results specifically using 8 0.5, which is a nonoptimal choice for 6, as we will show in the sensitivity analysis in Appendix E.I.
We visualize the corrupted distribution and its rectified distribution when the corruption level is 45% in Figure 6. In this case, 55% of the data is drawn from the true distribution N NO, 2) and 45% is drawn from (25, 2). Figure 6(a) displays the original sample of data from the contaminated distribution. The outlier points from the contaminating distribution are clearly visible in orange. Figure 6(b) shows the rectified distribution our estimator produces for d 2.5, in which the outlier points have been moved from their original values to their rectified values, which is much closer to the true mean of the clean distribution. Our estimator thus successfully identifies the majority of outliers and relocates them towards the center (our mean estimator), providing further support for our theoretical findings in the previous section. The sensitivity analysis of mean estimation with respect å is displayed in Figure 6(c). In this figure, the loss on the clean data is plotted for various values of 8. We see that an approximate minimum occurs at 2, with good performance within the range 8 [0.5, 2.5). This illustrates the relative insensitivity of our estimator to different values of ő for a wide and reasonable range. This range can be easily reached by hyperparameter tuning via cooss-vlldattin,, as the function describing the performance of our estimator is approximately quasiconvex. Moreover, when 5 = 0 or when ó set so large that it can rectify all points, our estimator gracefully degrades to the loss the median estimator, which is another favorable property. A further sensitivity analysis with respect to r is given Appendix F.2. In this sensitivity analysis, we find a similarly wide region of good performance for which improves significantly and almost uniformly on the more typical setting of r
Setuliily andyysh decred
(a)
(b)
(c)
Figure 6: (a): Visualization of contamination model: a mixture of Gaussian 0.55 x ( 2) 0.45 x (255, 2); (b): The rectified data generated by the proposed statistically robust estimator; (c) The sensitivity analysis of 8. The visualization of (a) and (b) shows that the outlier points are rectified into the bulk of the clean distribution at 5 2.5. The sensitivity analysis of 5 shows that this rectification occurs for a wide range of 8 which leads to losses for mean estimation which are lower than that of the median estimator of location. As 8 - 0 and increases, the performance of our estimator gracefully degrades to that the median estimator.
The evolution of the rectified distribution for mean estimation under concave and convex cost functions as increases depicted in Appendices F.3 and F.4. These depictions show that our statistically robust estimator which applies the concave transport cost function acts in stable and
22

---

expected manner as 8 increases. In particular, this depiction shows that our estimator moves points in the order of their distance from the estimate of the mean: that is, the budget is "used" on the worst outliers first. This results in an orderly procedure of rectification which is stable and predictable across nearby values of which a favorable property if is being set sequentially or cross-va vooss-validttion. In contrast, the estimator which applies the convex transport cost function moves all points of the clean and outlier distributions toward each other, which results in poor estimates of the true mean and an improper rectified distribution.
E.2 LAD Regression
As we discussed in the previous result, the theoretical analysis for the LAD estimator resembles that of the mean estimation task. Empirically, we also observe similar performance for the LAD estimator and show that our estimator correctly rectifies most of outliers to the fitted hyperplane under a range of choices for 8. In order to further support the effectiveness of our estimator, we provide additional visualizations: Figure 7 contains a visualization of the lines of best fit produced by various estimations to the LAD regression problem, and displays the effect of different choices of on the line of best fit produced by our estimator. In Figure 7(a) our estimator with 6 = successfully produces a line of best fit which is closest to the uncontaminated distribution, while the other estimators (OLS, LAD, and Huber regressors) produce lines which are heavily affected by the contaminating distribution, showcasing the robustness of our estimator. Figure 7(b) and 7(c) show the rectified distribution produced by our estimator under = 1 (b) and 6 = 1.5 (c). The suboptimal value of 1 still rectifies many points from the contaminated distribution and produces good line of best fit. Setting 5 = 1.5 rectifies all of the points from the contaminated distribution and essentially recovers the true line of best fit. Importantly, this simulation shows that even improperly setting 8 to the suboptimal value of 5 = 1 produces a much better line than any of the other estimators in Figure 7(a). Appendix F.5 contains comparisons over different random trials and sensitivity analysis with respect d and r, along with experimental results.
Somupted en tas 45% compted
Raca Tactfled &
Netttied Dotrbtatien 38,r-
Chars
Cloe Saca
Can Bun
Car Bus
(a)
(b)
(c)
Figure 7: (a): Visualization of contamination model: 55% are drawn with x ~ N/ 2) and 0.1 0.8r + 0.2w. 45% are drawn with I ~ N44, 2) and y = 10.1 - 0.8x + 0.2w. In each case w (0, Fitted models for different baselines are: OLS (ordinary least square). LAD (least absolute deviation regression). Huber (Huber regression with threshold parameter 1.5); (b): The rectified data generated by the proposed statistically robust estimator with small budget 8 1; (c): The rectified data with a larger ö = 1.5 is able to rectify all outliers.
23

---

F Additional Mean Estimation and LAD Regression Experimental Details
F1
Illustrative Example Details
In this example, we perform linear regression using our estimator with either a concave or convex transport cost function. The estimation is performed on a sample of points in I2² drawn from a contaminated distribution (corresponding to P') where 45% of the points an outlier distribution and 55% of the points are distributed according to the true uncontaminated distribution (corresponding to P... Our goal is recover the line of best fit which corresponds the uncontaminated distribution, which is = 0.1 - 0.8r. In other words, 55% of the points are drawn with x ~ (0, 2) and y = 0.1 0.8x + 0.2wr, and 45% of the points are drawn with x NN 2) and y = 10.1 - 0.8r + 0.2w. We illustrate the considerable differences which can occur when using each of these cost functions under various choices of 6, the rectification budget parameter. We depict this example in Figure 2 for the concave cost and Figure 3 for the convex cost.
distributed according to
The behavior of the estimator using the concave cost function is depicted in Figures 2(a)-(c). Each of these subfigures shows the points drawn from each distribution, the rectified distribution, the true line of best fit for the uncontaminated distribution, and the line of best fit produced by our estimator, at various choices of 6. In Figure 2(a), = 0, in Figure 2(b), 5 = 0.9, and in Figure 2(c), 6 = 1.5. As note that the proposed estimator using
can be seen from the figures, the budget 8 increases, the concave cost function becomes increasingly adept at mitigating the influence of outlier data. It achieves this by rectifying the outlying points, moving them into the bulk of the uncontaminated distribution. As a result, the line of best fit (depicted in red) aligns much more closely with the true line of best fit (depicted in black). It is able to perform this successful estimation because each outlier point can be moved far with a cheap cost (a "long haul"") due to the use of the concave transport cost function.
However, the convex cost function may result in suboptimal estimators, which can only move every data point (clean and contaminated) little bit. Figures 3(a)-(c) display the behavior of the estimator using the convex cost function, which illustrates this effect. Each subfigure displays the rectified distribution as yellow-green points for different setting of 8 (i.e. (a) 5 0, (b) 5 0.9, and (c) 5 = 1.5). As can be seen from the evolution of the rectified distribution, as 5 increases, instead of the outlier points being moved towards the bulk of the distribution, all of the points move towards each other-even the points in the clean distribution. This is not ideal behavior, as the points from the clean distribution should stay in place. This defective behavior causes the line of best fit produced by the estimator using the convex cost function (which is depicted in red) fit to the rectified points to be severely affected by the outliers; instead of being close to the ground truth line of best fit (which is depicted in black), its slope is moved significantly upwards towards the outliers. This occurs because the convex cost function gives lower cost to smaller rectifications of the given data set, which an inappropriate assumption for data sets containing outliers.
Appendices F.6 and F.7 depict the evolution of the rectified distribution as ō increases for the concave and convex cost functions, respectively. For our estimator which uses the concave cost, the evolution increases shows that our estimator increasingly rectifies the points in
of the rectified distribution order of their distance from the line of best fit in a predictable and stable way, which is a favorable property for setting 5 via hyperparameter tuning. In contrast, the convex estimator consistently moves all points (clean and corrupted) toward each other as 8 changes, which achieves poor estimate of the true line of best fit for all 8 considered.
E.1.1 Non-clustered Outlier Illustrative Example
In this section, we illustrate the performance of an additional experiment under different model for outliers. Like in the prior section, our goal is to recover the line of best fit which corresponds to the uncontaminated distribution. In this setting, we add additional terms to our data-generating process which cause heteroskedasticity and outlying points in the outlier data. In this setting, points are drawn from the clean distribution and contaminated distribution with the same probabilities (55% and 45%). We generate new random values for the data- eetta-ennrratng process: clean points are drawn with x ~ (O, 1) and y = 0.4 + 3.5x + 0.2ur, where w *N(00) 1). Contaminated points are drawn according to ã ~ (0, 1) and x = T + min("? (20) + and y = 10.4 + 4.5x + 5w. We illustrate the estimated lines of best fit under various choices of ò, the rectification budget parameter, Figure 8 for the concave cost, showing that our estimator still performs well against non-clustered, heteroskedastic
24

---

outliers. As the value of the budget parameter 8 increases, the line of best fit estimated by our method approaches the line of best Notably, our estimator not entirely corrupted by the "outliers in the outliers", unlike the other benchmark methods.
Con
compptec 02.r+85
Campgted Diribuadon 10% comoredd 2i65
Dirmbbtton compted eid5
Ola
dack
Cloe Bace
(a)
(b)
Figure 8: Visualization of the performance of our approach on non-clustered, heteroskedastic outliers, with values of the budget parameter (a) 0.5, (b) 1.0, and (c) = 1.5.
F.2 Mean Estimation Details
The experimental details for mean estimation follow. For each (6, r) point in experiments, perform 100 random trials at 100 fixed seeds. In each trial, we run our optimization procedure for with learning rate of 10-2, We stop when the number of iterations reaches 2000 or the change in the loss function between successive iterations is below a tolerance of 10-6, We initialize 0 to the median of the data
Below in Figure 9, we visualize the sensitivity analysis of r at 8 = 1 when the corruption level is 45%. As shown, our concave transport cost function improves significantly on the linear cost function (r = 1) and has a wide region of favorable stable performance.
Sensitivity analysis of = (451% Corrupted Data)
2.8
2.6
2.4
2.0
1.8
Mean Loss
Standard Deviation
1.6
Linear Transport Cost (r 1)
0.0
0.6
10
Figure 9: The sensitivity analysis of the loss clean data with respect of our estimator 5 = 1 on data with a 45% corruption level on the mean estimation task.
25

---

F33 Concave Cost Mean Estimation Simulation
In Figure 10 below, we plot the evolution of the rectified distribution produced by our estimator under various values of 8 for the concave cost function with P 0.5.
Tacti ad Dirribttoo. d- Eur 0.5 (55% Gompaad
Factiedd Setrittion di 0155. aud 115% Carupidd
Fnd Didoar 4- 2-2. 2.5 125% Camatted
(a)
(b)
(c)
Cuc 2.6% 2.5 1515% Compatd
Sucion, E.5 Carrated
Racifed Merrictionn 6.% (- 33% Camunadi
A
(d)
(e)
(f)
Gorsptadt
Sea
(g)
(h)
ii)
Ged Catrbitin 5- 1.33 E5 155% Compate
Fert fot Dis Botion 3- 1.5, P- 12%% Camptted
Taceled Dimiittion de 1,00 -03 13% Comprrot
50
3.00
i
(k)
Figure 10: Visualization of the evolution of the rectified distribution.
26

---

F.4 Convex Cost Mean Estimation Simulation
In the Figure 11 below, we plot the evolution of the rectified distribution produced by our estimator under various values of å for the convex cost function with r = 2.0.
Bostlfed Diitri stteerter.cco duet 2.13% Compttel
lecilled 20tra 0-0333 C33 Comsated
Poctiet Dialiiud 9- 2. 12%% Comptted
21
(a)
(b)
(c)
Eit
(d)
(e)
(f)
Derbaakn d- Gomgaadd
hcciled Sark de L40 and Camsated
Rectial Dialied 2 160% Corpetdd
(g)
(h)
Tact Not Doribction 5- 1.35 2 115% Oemagoott
Factilet Distiot d- 3. 125% Compoted)
Tectiteed Debrit 0-1.85 C3W Compated
10
36
(i)
(k)
(1)
Figure 11: Visualization of the evolution of the rectified distribution.
27

---

F,5 Least Absolute Regression Details
We provide additional results on the least absolute deviation simulations below. In Table we report the results of our experiment. We take the same experimental setting as in the regression example from the main text.
For each (ó, r) point in our experiments, we perform 100 random trials at 100 fixed seeds. In each trial,
optimization procedure for with learning rate of 10-2 starting from 10 randomly initialized points and take the best loss. In each problem, we stop optimization when the number of iterations reaches 1000 or the change in the loss function between successive iterations is below tolerance of 10-6, We initialize the slope /3 according to a N N(0, 1) distribution and the bias to zero.
The results included in Table 4. As can be seen, our estimator outperforms all competing estimators, and attains significant outperformance for most corruption levels.
Table 4: We compared our estimator with several standard regression methods by evaluating the average loss on clean data points across various corruption levels. Mcan performance is accompanied by a 95% confidence interval over 100 random trials. In our evaluation, we the threshold parameter of Huber regression to 1 1.5. The hyperparameters for our estimator, namely 5 1.5 and 0.5. remained constant across all corrupted levels. Error bars represent two standard deviation confidence intervals we have computed manually assuming normal errors.
Corruption Level
20%
30%
40%
45%
49%
OLS
1.569 + 0.041
1.702 + 0.043
1.773 0.049
1.803 0.052
1.822 + 0.054
LAD
1,808 0.131
1.875
1.892 0.055
1,903 0.066
1.908 - 0.006
Huber
1.642 0.042
1.842 + 0.053
1.868 + 0.056
1.882 + 0.059
Ours
0.657 = 0.638
0.529 0.430
0.680 0.467
0.802 0.597
0.866 0.619
In Figure 12 and Figure 13 below, plot the sensitivity of our estimator across different values of 5 and r. We see that there is a reasonable basin of good performance across both values. Each loss curve approaches the error of benchmark estimator or estimator with traditional cost function let and r tend toward values which recover these approaches.
28

---

Sensitivity analysis 0.5
1.75
Loss clean data
1.50
1.25
0.75
Mean Loss
0.50
#2 Standard Deviation
Median Estimator Loss
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Figure 12: The sensitivity analysis of the loss on clean data with respect to á of our estimator at = 0.5 on data with 45% corruption level on the least absolute regression task.
Sensitivity analysis of r, 6 1.5 (45% Corrupted Data)
1.8
1.6
1.4
1.2
1.0
0.6
Mean Loss
Standard Deviation
0.4
Linear Transport Cost (r=
0.0
0.2
0.4
0.6
0.8
10
Figure 13: The sensitivity analysis of the loss on clean data with respect to r of our estimator at ô = 1.5 on data with 45% corruption level the least absolute regression task.
29

---

F.6 Concave Cost Regression Simulation
In the Figure 14, we plot the evolution of the rectified distribution and line of best fit produced by our LAD regression estimator under various values of 5 for the concave cost function with r = 0.5.
Salif plac batke doeteweiia dil Lotns Dunggaol
Pasificl tookk Aos owntls with 100% Consored
Pallie lea Hiieda owstch a lays Donazedi
pivui But
Diued
(a)
(b)
Netiiea of ledten tww 33 rw Sarated
Katifcl Mordiii Ow wi3 20H5 Comuustt
Pasilc Bookkiii Ow Lib, 33 Jan Can Del
Our
Oar beneteere
(d)
(c)
(f)
Pablicd JoH
Der
(g)
(h)
i
Neshial aallodo 333 1 1405 Dergtted
Pabifc Diadkcioon Ohd LAO, 13 J0Y5 Conaurll
Paoiled Diadbsioon Ond rat 40% Danoxed)
Oivil Bu
Dbuid
i
(k)
Figure 14: Visualization of the evolution of the rectified distribution.
30

---

F.7 Convex Cost Regression Simulation
In the Figure 15, we plot the evolution of the rectified distribution and line of best fit produced by our LAD regression estimator under various values of 5 for the convex cost function with r = 2.0.
butifod iiir sleee cedlure 240r Lurophel
fied Dey kuider SwiL and 1400 Caredted
Fri fiie Uii kaikes bwi00
lac
Soak sub
pivud Eut
Diued
(8)
(b)
Pos Boriaaoon bw r 22V6 Oengred
Butifed 20rk duk ow Caregted
Sotifed b0rk 33, a2 (43% Comaated
Our
(d)
(c)
(f)
Camsted
Doke Eul.
Coeer
(h)
i
Rer Boollatoon Aw 1.0, ea J2SS Ourggell
Koiifed 20r dike dw a (40% Carested
Sac Droiktion da 2n (4%% Careded
Coel ara
Svan Tulh
Doudd
i
(k)
Figure 15: Visualization of the evolution of the rectified distribution.
31

---

G Additional Volatility Modeling Details & Experiments
G.1
Additional Volatility Surface Experiment
We have performed an additional experiment to demonstrate the practicality of our method and the selection of the parameter å with the realistic options outlier data set of our empirical study. This experiment shows that our method leads to better out-of-sample results on complex data and does not require the availability of clean data. The problem is the estimation of the volatility surface in the presence of outliers for the option data.
Our experiment proceeds follows. In this experiment, the data set is organized list of (train day, test day) tuples. Train and test days are consecutive days for the same underlying security. On the train day, estimate our model with different methods. On the consecutive trading day (that is, the test day) we use the estimated model to obtain sut-ol- evaluation metrics for the surface (MAPE and V Š). Note that this approach follows closely how volatility surfaces are used in practice by traders. This allows us to obtain the out-of-sample MAPE and V S as a function of 6. The prior Appendix section provides the details of our empirical setup.
To perform cross-validation, we split the train day into a training and validation sample, which we use to obtain estimates of MAPE and VŠ as function of 8. We select the delta that optimizes MAPE and V and fit on the entire train day. Then, on the out-of-sample test day, we evaluate our method for the optimally selected Thus, to clear, for the estimated optimal ō, we estimate the surface on one day, and evaluate it completely out-of-sample on the consecutive trading day.
The results collected in the table below. The results of experiment show that approach outperforms competing approaches and that cross-validation improves upon the fixed ó results reported in the main text. The below tables show the out-of-sample MAPE and Vš averaged over all out-of-sample test days.
Model MAPE
0.5% Quantile
5% Quantile
Median
Mean
95% Quantile
99.5% Quantile
KS 2SKS
0.047
0.088
0.236
0.274
0.563
1.033
0.036
0.065
0.172
0,216
0.504
0.999
Ours 10 10-2)
0.037
0.065
0.170
0.203
0.440
0.841
Ours (CV)
0.036
0.064
0.170
0.203
0,437
0.837
Model V
0.5% Quantile
5% Quantile
Median
Mcan
95% Quantile
99.5% Quantile
KS 2SKS
0.323
1.616
14.042
19.653
59.534
105.361
0.036
0.121
1.759
6,823
30.020
76.652
Ours (5 10 3)
0.035
0.112
1.579
5.880
25.521
70.708
Ours (CV)
0.033
0.104
1.248
4.522
19.645
55.261
Table 5: Median and quantiles of the distribution of MAPE V across all samples for KS, 2SKS, and our methods. Our estimator achieves significantly lower MAPE and VŠ than the benchmark KS estimator and improves upon the 2SKS estimator. Our CV procedure improves even further.
32

---

G.2 Kernelized Regression Experiment Details
Data set. Our data set derived from European-style US equity options implied volatilities from the years 2019-2021, The data come from the OptionMetrics IvyDB US database accessed through Wharton Research Data Services (WRDS). Implied volatilities created by OptionMetrics are calculated using the Black-Scholes formula using interest rates derived from ICE IBA LIBOR rates and settlement prices of CME Eurodollar futures. Option prices are set to the midpoint of the best bid and offer quoted for the option captured at 3:59 PM ET.
We developed our options surface estimator code in partnership with a major global provider of financial data, indices, and analytical products. This provider had previously identified 1,970 outlier- containing option chains for which they found IVS estimation challenging. These contiin-on option chains were selected from all listed US equities on days within 2019-2021, and they comprise the data set used in our experiments. Because the code proprietary, and the data set a proprietary selection of surfaces, we cannot publicly release them. However, the data can be found in WRDS, and we provide all implementation details in this appendix section.
Implementation. We implement the KS estimator and our estimator in PyTorch. For the 2SKS method, options in an options chain are removed if their implied volatilities fall outside of the region [90.25 1.5 R. q0.75 + 1.5 +..-IRR where q0.25 the 25% quantile of the implied volatilities the options chain, 70.75 is defined similarly, and IQR is the interquartile range of the implied volatilities in the options chain. The surface is then estimated upon the remaining options via the KS method. For our estimator, we estimate the surface by subgradient descent with learning rate a = 10-1 and r 0.5, terminating when the relative change in loss reaches 10 10-5, We denote the test data set's option's implied volatilities v/ and the estimated surface's implied volatility for option as S(xz)). We begin each iteration by setting 8 = {xx, v) /2|"'' where é is the loss function of Theorem 4. We initialize / to those of the benchmark estimator. We perform 5 trials per options chain. In each trial, we randomly select a different 80% train and 20% test set, estimate the surface the train set, and record the two losses (MAPE and VS) on the test set. We depict all losses gathered in this way via the histograms of Section 5.2. Experiments are run on a server with a Xeon E5-2398 v3 processor and 756GB of RAM.
Cross-validation. The cross-validation (CV) procedure is standard. We sample 4/5 of the training surface as CV training set and leave the remaining 1/5 as the CV validation set. Five such splits are made and used to estimate the MAPE of ó E {10-6, 6 100- 100 100-- 10-2 10-1 1, 2, 5, 10). The MAPES of each 8 are averaged across the five CV runs and the 8 with the lowest MAPE then used to fit our estimator on the entire option chain.
Losses. These losses are chosen for their importance in finance and option trading and valuation. MAPE a preferable error metric for volatility surfaces, as option chains contain options with implied volatilities which can differ in orders of magnitude, and MAPE weighs equally the contribution to error of options with such volatilities. The discretized surface gradient V measures surface smoothness of the estimated volatility surface by computing discrete gradient across a grid of points in the (time to expiration, strike price) plane. Smooth surfaces are important for several reasons: (1) smoother surfaces allow more stable interpolation or extrapolation of implied volatility for options with market prices which are not directly observable; (2) smoother surfaces produce smaller adjustments for option hedging positions constructed from measures of the IVS, which ultimately lowers hedging transaction costs; (3) smoother surfaces are less likely to have internal arbitrages between options, which are implied by sharp discontinuities in the IVS. putting the surface more in line with established asset pricing theory; and (4) smoother surfaces produce more stable estimates of financial institutions' derivative exposures, which are required to be consistent for financial regulatory and reporting requirements. These measures are standard in options IVS estimation. We define the option implied volatility surface $(x) as the function from option feature vectors estimated implied volatilities y. That is, for some given ær we have y == S(x). The accuracy measure MAPE is defined as follows for some test set of options ((0), v() 1
1
((x() - v'l 1vAl +
MAPPE(S. (ol
33

---

where e is a small numerical stability factor we set to 0.01. This is suitable choice, as 999% of implied volatilities are greater than 0.1.
The smoothness measure V is defined follows:
(Smm1 ST, + (ST, )T,,,))
VS
2
1= 1=
In the expression above, Sr.A, S(log T., u 2) where = 1 if A; > 0 and 0 otherwise. Here, each tuple (Ti, Ay) is a member of the Cartesian product of [rç : [p] and {Ax : E [s]]. where "[m]] denotes the set of natural numbers from 0 m, and p 10 and 39. By convention, OptionMetrics selects the following 11 discretization points for т, which we follow:
7 E {10, 30, 60, 91, 122, 152, 182, 273, 365, 7303.
For A, we use the following 40 points, which are a superset of the set of discretization points which OptionMetrics selects:
A E 4-1.0 : m E [40]} {0).
Example implied volatility surfaces. the following surface figures, we display the significant outlier rectification effect of our estimator on sample option chain which contains outliers. These examples serves to illustrate the efficacy of the automatic outlier rectification mechanism and to provide intuition for the surface fitting problem.
34

---

1.0
0.20
0.5
100
0.1
150
0/00
200
Days USI arpicatin
300 11.00
Figure 16: This plot depicts the option implied volatility surface estimated by the benchmark KS method on the options, which are depicted as blue and red dots. Blue denotes call options and red denotes put options. Outliers are present in this option chain near 0-100 days until expiration for deltas around A -0.2 (approximately four put option outlirss and call option outlier) and 1.0 (approximately 3 call option outliers). These outliers heavily corrupt the fitted surface, wildly distorting the values around these deltas and causing a poor fit for surrounding options which are not outliers. The surface reaches values of 90% annualized implied volatility and has surface gradient V of 1.48.
35

---

1.00
2.6
0.00
2.56
100
0.3
100
0.0
200
1.29
Days UnS R1PeR000
-1.00
Figure 17: This plot depicts the option implied volatility surface ((x) estimated by estimator with 1 on the displayed options, which are depicted as blue and red dots. Blue denotes call options and red denotes put options. Outliers present in this option chain near 0-100 days until expiration for deltas around -0.2 and 1.0. However, these outliers do not corrupt the fitted surface, which fits options nearby the outlying options quite well compared the surface fit by the benchmark method displayed in Figure G.2. Additionally, in contrast to the surface fit by the benchmark KS method, the surface fit by our estimator only reaches values of 65% annualized implied volatility and has surface gradient V of 0.69, smoothness improvement of over 100%. This example illustrates the capability of our estimator to rectify outliers.
36

---

G.3
Deep Learning Experiment Details
The full details and description of the deep learning local volatility surface estimation experiment are given in this section.
Background. IVS estimation provides an approximate surface fit for the volatilities implied by option market prices. However, market participants often require surfaces which fit as closely as possible to the market-implied volatilities.' ! One of the most popular methods used in mathematical finance to estimate surfaces obeying such a constraint under a reasonable stochastic process model for the evolution of asset prices the local volatility or implied volatility function model introduced by Rubinstein [1994], Dupire et al. [1994], and Derman et al. [1996]. The local volatility model assumptions imply that a function o (K 7) exists which gives an analytic expression for the volatility a at a given option strike price K and option time to expiration T. This function is known as Dupire's 's formula after its originator. Enforcing Dupire's ' formula in the estimation of a surface results in a local volatility surface, which is of great use to financial market participants. However, estimating such a surface becomes significantly complicated if market data contains corruptions or unrealistic outliers which are unlikely to occur in the future. 2 As human intervention for outlier rectification in large number of IVS fitting tasks is unrealistic, an automatic method for rectifying outliers in models fitting local volatility surfaces is required.
Data. We utilize the data set of Chataigner et al. [2020], which contains 17 (options chain, IVS) pairs taken from the German DAX index in August 2001. Options chains are captured from daily options market data, and IVSes are estimated for grid of strikes and maturities via a trinomial tree calibrated by the method of Crépey [2002]. In addition to Chataigner et al. [2020]. this data set is also used in Crépey [2002] and Crépey [2004]. In this data set's usage in the literature, the authors train and test using the price surface, which is given by nonlinear transformation of the IVS. Consequently, we also use the price surface for fitting; however, we report results after transforming back to the IVS. To corrupt the data set, a percentage of the options in each options chain are selected. We refer to this percentage, the corruption level, as E. The prices of these options x are then corrupted by replacing them with the values 10z.
Benchmark Method. As a benchmark, we apply the state-of-ther arr deep learning approach from Chataigner et al. [2020], the Dupire neural network. In this approach, a neural network estimates the price surface under the local volatility model assumptions encoded in Dupire's s formula, and additional no-arbitrage constraints which are useful for empirical applications. These conditions are enforced using different approaches: hard, split, and soft constraints. Hard constraints enforce the conditions by separating the network into subnetworks which have separate input layers for the variables involved in each of the conditions; conditions such as convexity and non-negatie non-negativity are then enforced by projection steps during optimization and the proper choice of activation functions. Hard constraints thus learn a function which explicitly satisfies the conditions. Soft constraints enforce these conditions instead by penalizing violations of them each of the training points observed from the market. Hybrid constraints split the network into subnetworks which have separate input layers for each of the constrained variables as in hard constraints, and utilize soft constraints for enforcing the Dupire and no-arbitrage conditions. The function learned only approximately satisfies the conditions in theory, but in practice the approximation very good (see Chataigner et al. [2020] for details). Because these approaches estimate the price surface, we also estimate the price surface in this application; however, the local volatility surface easily recovered from the price surface by a nonlinear transformation.
Robust Neural Network Estimation Problem & Implementation. To estimate price and volatility surfaces under the induced corruption, we estimated the benchmark approach using our statistically robust estimator. For each options chain, we tune ē via cross-validation: for each day, we sample 80% of the training set without replacement as a cross-validation (CV) training set, and use the remaining
For example, banks pricing some kinds of exotic options require such surfaces to avoid arbitrage opportuni- ties in surfaces that their internal trading teams may exploit, outlined in Hull and Basu [2022]. Participants pricing options may moreover simply prefer to accept the assumption that the market prices usually correct, and thus prefer closely fitting surfaces.
"This may occur due to low liquidity, which allows market manipulation or adversarial trading to produce unrealistic prices.
37

---

20% of the training set as CV validation set. We then train our robust neural network estimator on the CV training set with 8 E {10-1 11002 ..... 10 J. For each ò, we then compute the MAPE on the CV validation set. We then select the ê with the lowest MAPE CV validation set, and we use this ó to train our estimator on the entire training set. This produces our final estimated model. We base our implementation on the implementation of Chataigner et al. [2020] (see commit e03da98 at this url), which is BSD 3-clause licensed. The experiments were run on a server with a Xeon E5-2398 v3 processor with 756GB of RAM.
Results. maintain comparability with the benchmark, we test and report the same metrics reported by Chataigner et al. [2020], namely, the RMSE and MAPE for the surface estimation. The metrics for each day and model are repeated for three trials with different random weight initialization seeds. The results are displayed in Table 5.2.2. The first panel displays the RMSE and MAPE for each model using the different constraint techniques, averaged across all trials and corruption levels. The second panel displays the RMSE and MAPE for each corruption level averaged across all models. Our approach outperforms the baseline approach across all averages, and does so more clearly the corruption level increases, despite the strong regularizing effect of the no-arbitrage constraints and enforcement of Dupire's formula. Our improvement is present across all models, and especially impactful for the most accurate model utilizing soft arbitrage constraints. For this model, the out-of-sample test error in terms of RMSE and MAPE are reduced by 33% and 34%, respectively. This experiment displays the efficacy of our estimator state-of- of-ieo----h--tttttte deep learning model for option price surface modeling.
38

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA]-
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1 (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked While "[Yes] " is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providded a proper justification is given (e.g., "error bars expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
the checklist one of the factors in their evaluation.
reported because it would be too computationally
"[No] " or "[NA] is not grounds for rejection. While the questions are phrased acknowledge that the true answer often more nuanced, so please just use your best, judgment and write justification elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The paper claims to give an automatic outlier rectification mechanism that integrates rectification and estimation within a joint optimization framework and demonstrate effectiveness experiments. These claims are satisfied in the paper via definitions, theorems, proofs, and experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
39

---

Answer: [Yes]
The paper clearly summarizes and refers to the appendix for limitations to our work.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All important equations numbered and cross-referenced, and assumptions, lemmas, and proofs are formally given in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
40

---

Justification: All information required for experimental replication is detailed in the experi- mental appendices.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The work in this paper was conducted with an industry partner, and the code and data are proprietary. However, most of the experiments are reproducible, and experimental details, algorithms, and data sources are clearly stated.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
Seaidess/oodSuumiisee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: //nips.co./. wwtpccppppppppttccccccccccccteet httpe///iideeeeeeeuuuiiissssses http://oeeeuumiiiiiooooooo.ooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
41

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We clearly state all experimental details in the appendices.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Error bars or quantile information are included in tables, except for the deep learning experiment, which is expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details provided in the experiment appendices.
42

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compule required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http:///errips..eeoo htce/public/Ehhicsoooooe ines?
Answer: [Yes]
Justification: This paper complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper concerns work which is purely focused on mathematical and computational problem.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
43

---

Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Prior code, data, and models with licenses are clearly cited in the appendix.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
44

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This paper did not conduct crowdsoureing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: This paper has no crowdsourcing or human subject research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
45


Reviews:
[{"review_id": "qfAB72bmtY", "reviewer": "Reviewer_fFC9", "summary": "A new approach for robust estimation is proposed based inspired by optimal transport. The general approach is introduced and then new estimators are derived for three particular cases : mean estimation, linear fitting and surface regression. The estimators are compared numerically with standard estimators on synthetic data for the first two cases and on financial datasets for surface regression. Many details, proofs and explanation are provided in the appendix which is 25 pages long.", "questions": "Is this works may help to interpret and better understand the performances obtained by back-propagation during learning ?\n\nDetails :\n\npage 5, line 200 : where is delta in problem (3) ?\npage 6, line 209 : is it equation (4) or (3) ?", "limitations": "The proposed approach being very generic, the limitations are quite reduced. There are probably no potential negative societal impact.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The proposed approach seems very original and interesting. A new robust estimator may have a major impact on all the learning research field.", "weaknesses": "The paper is well presented and explained but the author material is so large that an article is not enough. There is probably material for a book. This raise the question to publish a paper with so large appendix ? The link with neural networks needs to be reinforced.\n\nA few details :\n\npage 4, line 160 : reference missing.\npage 6, line 227 : distirbution => distribution", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZnMEnPRybv", "reviewer": "Reviewer_i67o", "summary": "This paper is, essentially, about robust model fitting. In this paper, it is proposed to optimize an optimal transport problem. As loss function, the authors choose concave functions ||z-z‘||^r with r \\in (0,1). The idea behind choosing this shape of functions is that it allows to move outliers farther due to decreasing increases of transportation costs. The authors provide a proof that outliers can be identified easily by solving a linear program. Alternatively, they propose to use the quick select algorithm. Experimental results of the proposed algorithm are presented on option volatility prediction, a problem in finance.", "questions": "* What is delta in Fig 2? Is this the budget? If so, please mention this in the caption of the figure. \n\n* I would have liked to see more figures on the selected knot points and lambda, for instance some examples for which the proposed algorithms (LP vs QSelect) work well or fail.", "limitations": "I do think so. I believe that this algorithm constitutes an important contribution to the field and can inspire new papers by other researchers.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* I like that the paper presents a mathematically concise framework how to identify outliers in data. This is different from many practical algorithms that propose ad-hoc heuristics.\n* I like that the paper is written in a relatively accessible way.\n* The problem addressed here is important in many real-world problem.", "weaknesses": "* As far as I can can tell, the idea is novel yet somewhat similar to what has been proposed in this paper:\nChin et al., Accelerated Hypothesis Generation for Multistructure Data via Preference Analysis, TPAMI 2012\nIt would be nice if the authors could cite and discuss this paper.\n\n* The authors state in their abstract that the commonly used 2-step approach (first outlier removal followed by model fitting) does not provide information about the outliers to the model fitting stage. While that is true, I wonder which information and how their algorithm provides to the model fitting as there is no flow of information (for instance, by backpropagation) between outlier removal or knot point identification and the gradient step.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CFmvV3hhjX", "reviewer": "Reviewer_CDCT", "summary": "The paper presents a novel method for outlier rectification in robust statistics. In particular, taking inspiration from distributionally robust\noptimization (DRO) methods, this paper proposes to extend the formulation to the field of robust statistics. To this end, a rectification set is constructed using optimal transport distances with concave cost functions.  The authors have proven theoretically that the reformulation is eventually equivalent to an adaptive quantile (regression) estimator with the quantile controlled by the budget parameter for mean estimation and least absolute regression. \n\nExperiments are conducted to demonstrate the effectiveness of the proposed method on various tasks, including mean estimation, least absolute regression, and option implied volatility surface estimation.", "questions": "Please respond to the two points in Weaknesses.", "limitations": "I cannot find a discussion of limitations in the main paper or appendix.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Theoretically, the paper explores an interesting extension of DRO in the context of robust statistics. The introduction of optimal transport distance with concave loss functions is well-grounded. The resulting algorithm admits an intuitive explanation.", "weaknesses": "1. It seems to me that theoretically, the presentation is not particularly clear regarding how certain designs, such as optimal transport and concave cost functions, affect the final algorithm. At the core of the connection is Proposition 1 in the paper, but the proof is not provided.\n\n2. Experiment-wise, I find that the baseline methods are somewhat weak. The baseline methods in Sec 5.2.1 are very old, and in Sec 5.2.2 the baseline method does not consider robustness in estimation.\n It would be good if the adaptive quantile estimator is included for comparison. It is also advised to not put the main experimental results of Sec 5.1 in Appendix. \n\nOther comments:\nEq.3 is wrongly referenced (I believe it should be Eq.4) in multiple sentences, such as line 145. \nreference is missing in line 160", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dE5z0KYnms", "reviewer": "Reviewer_iniy", "summary": "The paper presents a new framework for outlier detection using optimal transport with a concave cost function. Unlike traditional methods that separate outlier detection and estimation, this approach integrates both into a single optimization process, improving efficiency. The concave cost function helps accurately identify outliers. The method outperforms traditional approaches in simulations and empirical analyses for tasks like mean estimation, least absolute regression, and fitting option implied volatility surfaces.", "questions": "Please see Weaknesses", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written \n- The proposed method is novel and theoretically sound\n- Extensive experiments are conducted to support the claims", "weaknesses": "- Could the authors discuss the relationship between the proposed method and M-estimator and RANSAC?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BGPSoEurU5", "reviewer": "Reviewer_qw3a", "summary": "A framework for regression tasks with outliers (see robust statistics) is introduced, based on an optimal transport cost function. A novel concave cost function directly facilitates the rectification of outliers. Mathematical proofs are given and evaluation is performed on two simpler and one real world task with state-of-the-art results.", "questions": "Q1: why are there no other \"robust\" variants of regular estimatiors used as baselines for the experiments? (i.e. mean estimation task,...)\n\nQ2: Mean&LAD (App.E): The tasks look relatively simple with two very clear clusters of inliers and outliers. Why are there no experiments where there are many outliers sparsely distributed across the data with high variance? These are the tasks I usually see in robust model fitting (i.e. for homography estimation in Computer Vision). Would the estimator struggle to transport many points different amounts of smaller distances, to stay within the \"long haul\" metaphore?\n\nQ2.1: The same goes for the Deep Learning comparison. Without knowing the variance of the price, going from p to 10p seems like a huge step. Was an ablation on the factor of the price (here 10) done?", "limitations": "- no potential negative societal impact exist\n- currently limited to regression tasks, although the authors point towards possible extensions to classification in Appendix B\n- evaluation is currently limited to the authors toy tasks and a few tasks from finance. the broad applicability of their method could have also contributed to more extensive experiments", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The theory and field of robust statistics is well explained and brought into context nicely with adjacent topics, with especially Fig. 1 and its explanation providing a good lens to view these problems\n- The paper is organized exceptionally well, despite its size (incl. the huge appendix)\n\n- All proofs in Appendix C are generally easy to understand, even without strong mathematical background. I was even able to thoroughly verify proofs in C.1 myself \n\n- All experiments are explained in great depth and detail in the appendix material, making both a surface and deeper understanding of all topics possible.", "weaknesses": "- Some of the tasks involve artificially constructing outliers (such as the cluster of outliers in mean estimation & raising the price p to 10 * p in the deep learning volatility experiment), which may not correspond to real-world data corruption\n\n- as the code/data seems to be proprietary, no publication is intended, which may hinder reproducibility\n\nTypos and Formatting Issues:\n- the authors falsely reference eq. (3), when really they want to link eq. (4) in lines, 145, 147, 149, 180, 183 (non exhaustive list)\n- missing citation in line 160\n- formula slightly clips into the text at line 156\n- typo in line 746: \"esstimator\"\n- typo in line 874: \"exporation,\"\n- wrong citation: Nietert et al \"Outlier-robust wasserstein dro\" appeared in NeurIPS 2023, not 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers for their positive feedback and appreciate the effort to review our work. We respond below to individual points.

## fFC9
+ Material, neural nets.
While there is significant material in our work, we note that other papers at NeurIPS have large appendices as well. As our paper introduces a new estimation framework and theoretical results, we didn't have space to explore links to neural nets in depth. We are investigating it in future work.

## i67o
+ Flow of information.
We respectfully disagree that “there is no flow of information… between outlier removal… and the gradient step.” Our min-min estimator is an alternating minimization procedure, where each step informs the next. The gradient step depends on outlier rectification, and information is carried over via parameter updates across iterations. This approach ensures more information flow between the substeps than a two-step method.

+ Chin et al.
We appreciate this addition. While both papers address contaminated data, there are fundamental differences in terms of goals (hypothesis generation vs predictive modeling), the conceptual problem, theory, and application domains. We view this work as complementary and will certainly discuss and cite it in our camera-ready version’s related work.

+ Figures.
We’ve included many robustness figures in the appendix showing performance for many delta and r values. Since knot points and lambda depend on these params, we refer to Figures 6, 8, 11, and 12 for a comprehensive view. If the reviewer has a specific case, we would be happy to add it.

## CDCT
+ Designs, Proposition 1.
We appreciate the feedback on how we present how e.g. the transport cost function affects the algorithm. Specific suggestions would help us address this point. Pages 4-5 demonstrate how concave vs. convex cost functions affect the algorithm. Appendix F provides further explanation. Sensitivity analyses throughout illustrate additional details. We hope these sections suffice but welcome more feedback. For Prop 1, we have given sources for the proof in the text following. The reviewer can find the proof in the cited work Blanchet and Murthy 2019, Theorem 1 and Remark 1.

+ Baselines.
We value the feedback and opportunity to clarify. Our estimation approach can be applied to any regression algorithm, and we demonstrate this through diverse methods in sections 5.2.1 & 5.2.2, including a SOTA deep learning estimator. We disagree that the deep learning estimator lacks robustness, as it incorporates different regularizations that impart robustness. Our method aims to robustify regression estimators, and we believe we show this. We’ll move results for 5.1 to the main text, as suggested.

+ Limitations.
We respectfully disagree that no limitations are addressed. App. D.2 and D.3 cover limitations of our procedure. In our camera-ready version, we’ll add a summary of these to the main text and reference these appendices.

## iniy
+ Relationships.
A full comparison is beyond scope, but we give an overview here and will include a version of this in our camera-ready version. M-estimators are a broad class of diverse extremum estimators containing robust and non-robust methods. Robust ones traditionally use specific loss functions to reduce outlier sensitivity. We believe our approach is more fundamental as it builds on top of a given general loss. The statistician has full control of the outlier modeling task (via specification of optimal transport theory using a concave transport cost) and the statistical task (via the chosen loss); also, our method gives the optimal rectifier (i.e. transporter). Our approach differs conceptually and demonstrates superior performance over the M-estimators in our experiments. We are open to additional M-estimators if suggested. RANSAC handles outliers by iteratively selecting random data subsets, fitting models, and evaluating models on inliers to reach a best model. In contrast, our method optimally selects the set of points for rectification using a single optimization procedure. RANSAC only identifies inliers, while our method does this and rectifies outliers. We believe our approach is more direct, potentially more computationally efficient. It's based on optimal transport theory rather than heuristic random sampling. 

## qw3a
+ Concerns.
We chose 10 to represent "fat finger" mistakes, such as decimal misplacements in finance, which have historically caused significant losses. We recognize the reviewer’s concern wrt proprietary code. Our method is used by industry partners with IP restrictions common in finance, but we've disclosed the algorithm, hyperparameters, and other details needed for replication. The deep learning data and baseline code are on GitHub.

+ Questions.
Q1: We use robust methods for mean estimation, e.g. the median and trimmed mean estimators. Notably, the trimmed mean uses the true corruption rate, a strong advantage. Despite this, our method, which doesn’t know any true parameters of the DGP, still outperforms this estimator. Q2: Our experiments for mean estimation and LAD regression are intentionally straightforward to make our novel framework easily understandable. Despite this, robust estimators like the trimmed mean and Huber perform poorly, indicating the difficulty of these problems and the improvement our estimator offers. Wrt deep learning, the choice of a multiplier of 10 is justified by the high variability in options prices, covering a wide range due to inherent price differences in options chains.

## All
+ Details/typos.
References and misspellings are fixed. References to Problem 3 have been corrected to Problem 4. The delta in Problem 4 is in the ball R(P’_n) in Eq. 5 (page 4 line 152). “Budget” has been added to Figs 2 and 3.

We hope the reviewers are satisfied with our responses. If our responses have adequately addressed concerns, then we politely request an increased score to reflect this change, if appropriate. Thank you!

Meta-review: I find that the joint approach to solving fitting as well as outlier estimation proposed has potential to influence the robust statistics field.
The paper's experimental validation could be further strengthened by studying broader class of outlier distributions. I am satisfied with the authors rebuttal and consider this to be a valuable paper to be accepted.
Decision: Accept (poster)