Robust Bayesian Regression via Hard Thresholding
Authors: Fan Zheyi
Zhaohui Li
Qingpei Hu
Venue: NeurIPS 2022 Accept (2022)
Keywords: Robust regression
Hard thresholding
Bayesian reweighting
Variational inference

Abstract: By combining robust regression and prior information, we develop an effective robust regression method that can resist adaptive adversarial attacks. Due to the widespread existence of noise and data corruption, it is necessary to recover the true regression parameters when a certain proportion of the response variables have been corrupted. Methods to overcome this problem often involve robust least-squares regression. However, few methods achieve good performance when dealing with severe adaptive adversarial attacks. Based on the combination of prior information and robust regression via hard thresholding, this paper proposes an algorithm that improves the breakdown point when facing adaptive adversarial attacks. Furthermore, to improve the robustness and reduce the estimation error caused by the inclusion of a prior, the idea of Bayesian reweighting is used to construct a more robust algorithm. We prove the theoretical convergence of proposed algorithms under mild conditions. Extensive experiments show that, under different dataset attacks, our algorithms achieve state-of-the-art results compared with other benchmark algorithms, demonstrating the robustness of the proposed approach.

Robust Bayesian Regression via Hard Thresholding
Zheyi Fan Zhaohui Li Qingpei Hu Hut.2t 1.2t
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, China School of Mathematical Sciences, University of Chinese Academy of Sciences, China H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, USA. 1,2 [fanzheyi, gingpeihul Camss . ac cn, Zzhohuui -ii@gatechhhh.... edu
Abstract
By combining robust regression and prior information, we develop an effective robust regression method that can resist adaptive adversarial attacks. Due to the widespread existence of noise and data corruption, it is necessary to recover the regression parameters when certain proportion of the response variables have been corrupted. Methods to overcome this problem often involve robust least-squares regression. However, few methods achieve good performance when dealing with severe adaptive adversarial attacks. Based on the combination prior information and robust regression via hard thresholding from [M], this paper proposes an algorithm that improves the breakdown point when facing adaptive adversarial attacks. Furthermore, to improve the robustness and reduce the esti- mation error caused by the inclusion of prior, the idea of Bayesian reweighting is used construct more robust algorithm. We prove the theoretical conver- gence of proposed algorithms under mild conditions. Extensive experiments show that, under different dataset attacks, our algorithms achieve state-of-the -ortth--rrtt results compared with other benchmark algorithms, demonstrating the robustness of the proposed approach.
Introduction
Least-squares methods are widely used because of their simplicity and ease of operation. However, due the inevitable existence of outliers, least-squares methods, such linear regression, may cause significant bias in practical applications. Therefore, to meet the challenge of learning reliable regression coefficients in the presence of significant corruption in the response vector, this paper focuses on robust least-squares regression (RLSR). RLSR has excellent application value in many fields, such signal processing [2]]] economics
[23], industry [21]], biology [[3],, remote
sensing [[ and intelligent transportation [25]].
Given a data matrix X [x1,... .... x,] E _dxn, the corresponding response vector R", and
a certain number representing the number corruptions the data, the RLSR problem can be described as:
(w. 5) = arg min WER? (CC, -
(Ue iES
x w)2
(1)
That is, we aim recover the correct point set S and the regression coefficient w simultaneously to achieve the minimum regression error. However, this problem is NP hard, so difficult to optimize directly [|[]]
To solve the above problem, a commonly used data generation model is y X w b* + 6, where w is the true regression coefficient we wish to recover and e is dense white noise vector
CCoresponding authors.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

subject to a specific distribution, that is, ||elo ~ n. The vector b' is k-sparse, which means that there are only non-zero values, representing unbounded noise terms in the response vector. After years of development, there are many ways to find a reasonable solution to the problem in Eq. (I). However, these methods typically only achieve good performance under specific conditions. The main challenge is the low breakdown point of conventional methods. The breakdown point k is a measure of robustness, which means the number of corruptions that the RLSR algorithm can tolerate. We can express k as the proportion of all data points: k a Many RLSR algorithms cannot guarantee theoretical convergence as the value of increases. For example, McWilliams et al. [[4]] used weighted subsampling for linear regression, but only had breakdown point of OlL/vd). Prasad et al. [[S]] proposed robust gradient estimator that can be applied to linear regression, but their method only tolerates corruption up to a = 0 d). Other methods may have higher breakdown point, but tend to assume specific pattern of data corruption. One representative adversary model for introducing data corruption is the oblivious adversarial attack (OAA), in which the opponent generates & sparse vectors while completely ignoring X, w", and 6. The work of Bhatia et al. [IT] and Suggala et al. [20] reported excellent results against OAAs by using a novel hard thresholding method; indeed, [20] suggested that a may even get close to I as n + oo. The recent online fast robust regression algorithm [[6]] also has consistent convergence with mild condition by using Stochastic Gradient Decent (SGD) algorithm. However, these methods cannot resist adaptive adversarial attack (AAA), in which opponents can view X, w", and before determining b'. Handling AAA challenging task, and so many methods can only guarantee a very low breakdown point, especially when the data distribution is not normal [[][)][[]]] Bhatia et al. [2] proposed the thresholding operator- based robust regression method and their breakdown point reach 1/65 for noiseless model i.e., c = O. However, their method can give consistent estimation only in noiseless case. Karmalka et al. [TC]] had a good result in sparse robust linear regression by applying the L1 regression and the breakdown point of their method reaches 0.239, but their estimation is consistent only when white noise is sparse. Diakonikolas et al. [[]] considered the situation that X and y may have outliers simultaneously, and proposed a filter algorithm in which the error bound is ((a log((/// (1/a)or). However, their method requires accurate data covariance of the true data distribution numerous unlabeled correct data to estimate the data covariance, which are often unavailable in practice.
The limitations of the
methods can be attributed to lack of prior knowledge from the real data, making it difficult to distinguish the set of correct points the case of AAAs Güllçhhre et al. [ showed that prior information effectively improves the accuracy of machine learning. In many application scenarios in industry, economics, and biology, prior knowledge such as previous experimental data or engineering data are available. The goal of this paper is to propose new robust regression that can integrate available prior information, even if the prior information is not very accurate.
The typical approach for integrating prior information is the Bayesian method. This provides a way of formalizing the process of learning from data to update beliefs in accordance with recent notions of knowledge synthesis [5]]. However, generic Bayesian method is also sensitive to outliers. Thus
robust Bayesian method should be considered to produce more reliable estimates in the presence of data corruption. Polson al. [[7]] used the local variance to assign each point local parameter that makes the estimation result robust. Furthermore, Wang et al. [p2] proposed local parameterization method and used empirical Bayesian estimation to determine the global parameters. Bhatia et al. [G] proposed a Bayesian descent method using an unadjusted Langevin algorithm (ULA), which guar- antees convergence in a finite number of steps. Wang et al. [22]] employed Bayesian reweighting to assign different weights to samples, thus reducing the impact of outliers.
In this paper, we combine the Bayesian method with hard thresholding method [M] and propose two algorithms, which we call TRIP and BRHT. Through assigning simple normal prior on the coefficients, TRIP can significantly increase the breakdown point when resisting AAAs. To fur- ther improve the accuracy of estimation, we propose BRHT algorithm through applying Bayesian reweighting method [22]] to coefficient estimation. Experiments show that BRIT is even more resis- tant to AAAs and gives lower estimation errors, demonstrating that our method achieves significantly improved robustness.
Our Contributions:
The main contribution of this paper is proposing new methods that combining the prior and robust regression to increase the breakdown point when encountering AAAs. We derive the theoretical guarantees given by our proposed algorithms. Compared with the consistent robust
2

---

regression (CRR) algorithm proposed in [T] we prove that our algorithms guarantee convergence under weaker condition, which also shows that our methods improves the breakdown point. We also establish an extended experiment to test the effectiveness of the algorithms. Compared with other basic algorithms, the experimental results show that our methods significantly outperform alternative methods under AAAs. Moreover, BRHT algorithm is also competitive against OAAs.
Paper Organization: We state the problem formulation and present some notation and tools in Section 2. In Section 3, we describe the details of our proposed TRIP and BRHT algorithms. The theoretical properties of these two algorithms are discussed in Section Section 5 presents extensive experimental results that demonstrate the excellent performance of our proposed algorithms. Section concludes this paper.
2 Problem Formulation
In this study, we mainly focus on the problem RLSR under AAAs. We are given a covariant matrix X [x,,..., x,.,,,]] Rdx" where X; E Rd. The true coefficient of the regression model is denoted by w". The response vector y E I is generated by:
= x" w''w"t + +
(2)
The perturbations to the response vector consist two parts: the adversarial corruption vector introduced by b", which is k-sparse vector, and the dense white noise ~ (O, o²). Our goal is to recover the true regression coefficient w" while simultaneously determining the corruption set To illustrate problem, we first pay attention to the standard robust regression problem in Eq. (II.
From the viewpoint of probability, the problem can be transformed into a log-likelihode version:
(w, $) arg
max
Σ log ((w
yis X1; 0²)
W ER SC(N))
We will try to convert the problem in Eq. (5) into a Bayesian version. From the Bayesian viewpoint, consider Pw(w) (w) as the prior of the coefficients in the model. In addition, add the localization parameter r and its prior (r) to reflect the change introduced by each additional sample. For any subset S [n]]. the distribution of all parameters and data X Ys is:
plys, w. Xs) Pw (w)pr(rs)
II (yv Tr x. SES
The posterior distributions p(w, xs and p(ys, w, rs| Xs) differ by regularization con- stant. We ignore this regularization constant in the posterior distribution and only consider the main terms of the parameters. We then formulate the Bayesian RLSR problem of searching for the subset and coefficients by maximizing the log- bog-posterior:
(w, 5) = arg
max WER' WER" PERT log (w) + Ellg Ely; | ri w, X,, x²) )++ + logpr(r:))
(5)
Sc|m),1s]=n-
RES
Note that we do not add any prior on 0² and only treat this as an adjustable parameter. This is because, in the initial stage of the algorithm described in Section EL the estimated o² will be large due to the existence of outliers, and this will make the estimation of w excessively biased to the prior distribution. This bias will be harmful, especially when the prior is sufficiently accurate. This phenomenon can be observed in Section B.I. To prove the positive effect of the prior on the RI.R problem, we require the properties of Subser Strong Convexity (SSC) and Subset Strong Smoothness (SSS). Given set S C [n]. Xs sx E Rdx[s] signifies the matrix with columns in the set S. The smallest and largest eigenvalues of square symmetric matrix are denoted by Amiu (X) and Amaz (X).
Definition (SSC Property). A matrix X E Bdxx is said to satisfy the SSC property at level m with constant Am if the following holds:
Am S min Amen Ammn(XxX3 (XsX3)
15
Definition 2 (SSS Property). A matrix X E Rdxn is said to satisfy the SSS property at level m with constant Am if the following holds:
max Amaz (XxX3) S Am
S
=m
3

---

Algorithm TRIP: hard Thresholding approach to Robust regression with sImple Prior
Input: Covariates = [x,,..., Xnl, responses = [y/1..., [Vii....Vul'', prior knowledge Wo.
penalty matrix M, corruption index k, tolerance
Output: solution w
1: b <- 0, 0, Pax t XT (XXT + +,)-1 PMPM & XT + M)-1M
2: while | - b' edo
3:
b'+1 f- HTM + PMxxly PMMWWO)
4: f 1=;
5: end while
6: return W (XX*)- bb)
These two properties are proposed in [2], and are intended to standardize the generation of the data matrix so that it will not be too abnormal. They are used to prove the theorems in Section HI.
3 Methodology
We first ignore the localization parameter in Eq. (5) and propose a simple method called TRIP in Section B.Il TRIP demonstrates the effect of prior on the hard thresholding method. To improve the robustness and the accuracy of the estimation, the BRHT algorithm proposed in Section B.Z.
3.1 TRIP: Hard Thresholding Approach to Robust Regression with Simple Prior
We propose a robust regression algorithm called TRIP (Algorithm M. a hard Thresholding approach to Robust regression with simple Prior. In this subsection, only the prior Pw (w) is considered and the localization parameter is not added to the model. We assume the variance o² of can be set by ourselves and that the prior (w) obeys a normal distribution ((wo, Eo), where Wo and Eo are determined in advance. Through the above simple parameter settings, the problem in Eq. (5) is transformed into the problem in Eq. (I) with an additional regularization term:
(w, $) arg
min wER" [S 1s|=n-k iES
Cw=--o w)² (w w M(w Wo)
(8)
where M (Eo/o?)-1 To solve this problem, proposed by Bhatia [I]]. which concentrated on recovering the errors instead of selecting the 'clean- est' set. The problem in Eq. (8) can be formulated as min (eR ||blloSk* 2 1| w (y b)
motivated by the hard thresholding method
[(w- wo) M(w - wo). Thus, if we have an estimation b of the corruption vector b", the estima- tion of w can be easily obtained by w (X + + 6) M wo]. By substituting this estimation into the optimization problem, we obtain a new formulation of the problem:
min f(b) = [I(MMx fy b) Parmwoll bll
where PMX-- = XT + -X) = ("XX + M)-MM. The hard thresholding step in the TRIP algorithm can be viewed as b HT' V (bb)), where & is the selected corruption coefficient. The hard thresholding operator HT; is defined as follows.
Definition 3 (Hard Thresholding). For any vector TR.", les 51) represent the position of the 7th element in which are arranged in descending order of magnitude. Then, for any k n, the hard thresholding operator is defined as f = (T), where i; = r; if 5. (i) < k and 0 otherwise.
The difference between the proposed TRIP algorithm and the original CRR [M]] the form of itera- tion step. The iteration step in both TRIP and CRR can be expressed uniformly as HT& - w²). w² (X + -x b') + wo] in TRIP and w' = (XX*)- The w' in CRR is just simple least square estimation, while the prior added in TRIP can be re- garded to adding quadratic regularization each iteration. This quadratic regularization can avoid the candidate of iteration that is too far from the prior mean, which is also helpful to ensure the
by) in CRR.

---

Algorithm 2 BRHT: robust Bayesian Reweighting regression via Hard Thresholding
Input: Covariates = =xx1,..., Xnl, responses = [V11.... Unl prior distribution pr(r). Pw (w)
corruption index k, tolerance
Output: solution W
1: b <- 0, 0,
2: while |]b' - b'-1 > do
3: w VBFM
b', ('PP(()),ww(M
4: bt+1 f- HT&(y X x=w
5: i f+t+l;
6: end while
7: return (Xx*)- b
Algorithm VBEM: Variational Bayes Expectation Maximization
Input: Covariates Xi..... X,l, responses y [1/1 .... ynl" prior distribution pr(r). Pw (w)
Output: solution w
repeat
2: update g(r)
3: update q(w)
4: until convergence
5: return MAP(g(w))
numerical stability of solution. Thus, as long as the prior mis-specified much, TRIP will be more likely to identify the uncorrupted points, and the final result of TRIP will be more robust than CRR.
Therefore, the prior plays an important role in TRIP. However, the weight of a prior in the solution depends entirely the matrix M (Zo/o²)-1 Therefore, if we estimation of 8² to replace o², or give o² a prior to calculate its posterior distribution, the overestimation of o² will cause a severe increase in M in the initial iteration steps. This will mislead the iteration and cause some deviation the final results. To overcome this difficulty we can directly treat M as an adjustable parameter control by specifying the form such M sI, where positive number, and the suitable parameter can be chosen through 5-fold or 10-fold cross validation.
3.2 BRHT: Robust Bayesian Reweighting Regression via Hard Thresholding
In this subsection, describe how the Bayesian reweighting method is combined with hard thresh- olding to give more robust algorithm, BRHT (Algorithm D), robust Bayesian Reweighting regres- sion via Hard Thresholding. We first introduce the reweighted probabilistic model (RPM) proposed in [24]] for traditional linear regression. For the covariates and the response y, the RPM model can be formulated as follows:
w//w PP
(w)(())p(f II C(vi w. s=1
(10)
where is the local weight assigned to each sample, Z is the normalizing constant, E(y: w,x, o²) represents the likelihood of the normal distribution A (x"" w.aa), and Pw (w), Pr (r) are the priors of and r, respectively. By ignoring the normalizing constant, the problem in Eq. (5) can be transformed into the following form under this RPM setting:
(w,S) $) arg
log Pw
PWgPw(W) (w) Irr log (1l/w, + logp,(r.) (11)
WERP PER"
The specific form of the prior (r:) be set to any nonnegative random variable distribution, including (but not limited to) the Gamma distribution, Beta distribution, or log-normal distribution. Here, still the normal distribution ((wo, En) the form of pw(w).
To solve the optimization problem Eq. (LI), we use the two-step BRHT algorithm. The key
iteration step in BRHT is b(+1 + HTxly X w where w' is calculated by maximizing the
5

---

rog-posterio of the RPM model:
(w", r²) = arg
max FERR
log (w) + lo + Σ re log E(vi - by x²) ==1
(12)
However, the direct inference of Eq. (I2) is hard because of the nonconvexity of this problem. In general, the parameters in this RPM model can be divided into two parts: the global variable w and the local latent variable r. To solve the inference problem of RPM, feasible method is to use variational Bayesian expectation maximization (VBEM). We set g(w, r) - [(())(((r) to approximate the true posterior after several iterations of VBEM (Algorithm (), and replace the estimation of w by the maximum a posteriori (MAP) estimation from the final approximate posterior. Full details of the VBEM method are given in Appendix A. It is reasonable to ask why we using Bayesian reweighting. Note that the iteration step in the TRIP algorithm is b < HTT (PMxb' P Pa wo) = (T& X X'w"), where w' (XX* + MX - b³) + Mwol Although we can show that TRIP already guarantees theoretical convergence, the estimation of w in every iteration still uses least-squares with penalty term, which is easily affected by the corrupted points. This disadvantage forces us to assign a higher weight to the prior to resist severe data corruption in the case of AAAs. However, a higher weight on the prior means a larger estimation bias. When applying the Bayesian reweighting method, the estimation in each step is more robust than the least-squares result, and thus the weight on the prior can be reduced guide the iteration. Therefore, the estimation bias is relatively small and the results are more robust. This is reflected in the experimental results presented in Section E.
We should also explain why we only use a prior for a few parameters. It is important to ensure that the prior weights are neither too high nor too low. As mentioned earlier, if we treat o² as the param- eter to be estimated, this places too much weight on the prior. We also ensure that Pw (w) does not create more uncertainty, such as setting Pw( to p(w, a) = [Ww a -EE (aa)Gam(aja aa, b,). Data corruption means that the subset of the training data may vary greatly from the prior information. Thus, when calculating the posterior, the variance of controlled by a will be very large to fit the data, and so the prior information wo will have lower weights in the inference step. The above prob- lems also mislead the estimation and the selection of subsets, so we do not consider the uncertainty of these quantities and simply treat them as model parameters to be set in advance. An adjustment method for all the parameters in BRHT is described in Appendix D.
4 Theoretical Convergence Analysis
In this section, we establish the convergence theory for the TRIP algorithm, and clearly explain how the prior effectively enhances the convergence of the RLSR model. Theorems 1 and 2 summarize the results. We also show the theoretical guarantee of the BRHT algorithm in Theorems 3-5, which further demonstrate the special properties achieved by using Bayesian reweighting. Before present- ing the convergence result, we first introduce some notation. Let A' := (Xx* +M)- - b*), g = (I Parx) c, and f := PM M (w" Wo). Let S. == [n] supp((") be the chosen subset that is considered to be uncorrupted, and I == supp(b') U supp(b').
Theorem 1. Let X = [xi,..., xn] E Rdx= be the given data matrix and y = X" + b* + be the corrupted output with sparse corruption of |lb'lo I k n. For a specific positive semi- definite matrix M, X satisfies the SSC and SSS properties such that 2,
(XXF < 1. Then, if k k*, it is guaranteed with a probability of least ô that, for any E, > !|b To - b l12 S
AEtk.
E + O(eo) + +0( +O(VAn V/ees Am)
Axxi || -ollz after To = Ollog( [bb]]a]] 6
iterations of TRIP, where
eo = 0 V0 (+)) log B(K+ke)) under the normal design.
Theorem 2. Under the conditions of Theorem 1, and assuming that x; E Rdd are generated from the standard normal distribution, if > k., it is guaranteed with probability of least 1 that, for any E, >> 0, the current estimation coefficient WT, satisfies l|Wr, w w2 0 ( ev)
0 Vk / Amax (M) ]| -olla after To = Olog( [lb'lla] )) steps.
n3/4
For positive semi-definite matrices XXT and M, Amin (XX" + M) Z Amin (XX") + Amin (M.). Agess
Thus, the condition 2
XXX < 1 in Theorem is weaker than the condition 2 Ac Xmenn (XXT) <

---

1 of Lemma 5 of Bhatia [I]] which shows that a prior can effectively improve the convergence of the algorithm. Assigning a higher weight to a prior means that M has larger eigenvalues, so that the convergence condition will be more easily satisfied As result, the TRIP algorithm can tolerate higher proportion of outliers than the CRR method of Bhatia (0]] and achieves higher breakdown point. In fact, under the condition limm (M) = 5. we can give an approximate expression of the breakdown point for TRIP when & is not too large: k* < & < (0.3023 - v0.0887 0.00400)m ), Details can be found in Appendix C.2. However, the improved convergence comes at the cost of an unavoidable reduction in precision. This can be seen from Theorem Z. If the data corruption is such that k- is (n) and the maximum eigenvalue of M is also O(n), then the bias of w cannot be decreased by adding more samples, which shows that there is trade-off between convergence and accuracy. A reliable prior improves both accuracy and convergence because it has a higher weight. However, an inaccurate prior can also be helpful as long as it is quite different from the distribution of outliers, and the convergence can be improved through a prior with a low weight.
To prove the properties of our BRHT algorithm, we define the following two intermediate variables to simplify the description:
U(w, r, 5) = =loo=loggae (eeppww)) (w) +
>>> Dlg (ra) rr log Elye | w, X, x2) YES
(13)
M(w. r, b) = (oogg(()) +
Σ (lg + + log Ely - by w. x,, 02)]
(14)
Theorem 5. Suppose that the prior of r; is independently and identically distributed (iid). We con- sider the 1h iteration step of the BRHT algorithm, where W, = arg max xERR M(w, , r, by)
and b. HT&(y w-)) is obtained from the hard thresholding step. Then, have that UW. U(W
Theorem 6. Consider a data matrix X and a specific positive semi-definite matrix M satisfying DEsk'
< 1. Then, there exist a > 0 and < < 1 where is small number, such that if k > k* and E in the prior Pw (w) is ao² M- it is guaranteed with a probability of at least 1 8 that, for any s, >> 0, |lb - -b* ll2 S s
the SSC and SSS properties such that 2 (XXT(XXX+MM
(cv) + +0( +O(AA (Aees--mmzlll eo O v (Is+ (K
[ill""-woll after To O(log(
0(log(2llb 2llb"l2)) iterations of BRHT, where
(XX'
k²) log 8(R+kT) under the normal design.
Theorem 7. Under the conditions of Theorem 4 and with x; E Rd generated from the standard
normal distribution, there exist a > O and < S 1 1+ c, where e is a small number, such thar if k k. and E in the prior Pw (w) is ao2M-1, it can be guaranteed with a probability of at least 1 - 8 that, for any E, >> 0, the current estimation coefficient WT, satisfies [[T, w= S 0 1 ]E + eo) 0 Vl++k Amax(M)
y/|l -woll after Tu Oloog( ([]b" steps.
n3/2
Theorem shows that our BRHT algorithm is reasonable because it optimizes the problem in Eq. (Il) in each step. Theorems B and @ guarantee the convergence of the parameter, which means that we introduce a prior Pw (w) = -V(w, Eo) to the TRIP algorithm, it is convergent. There then exists some >> 0 such that the prior Pw (w) (Nwo,, ooE) in the BRHT algorithm guarantees convergent parameters and the bias of the estimation of w will be y times that of TRIP. Note that is usually relatively large in practice, which causes lower prior weight in BRHT. Thus, when the convergence can be guaranteed, the bias of the estimator v can be significantly reduced because BRHT assigns lower weight to the prior. Even if the data are seriously corrupted, BRHT can ensure good results without significant error. All proofs of these theorems are given in Appendix C.
Experiments
In this section, we first consider how to effectively 'corrupt' the dataset using two different attacks: OAA and AAA. We then report an extensive experimental evaluation to verify the robustness of the proposed methods.
7

---

Algorithm 4 ADCA: Adaptive Data Corruption Algorithm
Input: Covariates = [X1, Xnl responses = [y1 Unl", true parameter w
penalty coefficient 5, corruption index k, tolerance
Output: solution w
1: b <- 0, 0. Pax < (XXX - 87) + (2 -87
2: while |bb - b'-1 do
3:
b'+1 t- (Tx(Psxb
Paxty + Pow")
4: i 1+
5: end while
6: W W+ -XX²)-1X((X b()
7: C supp(b')
8: return yc X
5.1 Data and Metrics
In experiments, the data generation be divided into two steps. First, generate the basic model. The true coefficient w" chosen to be a random unit norm vector. The covariant x; are iid in (0, Ta). The data are generated by Yi xf w" 6, where in the experiments. The second step is to generate the corrupted data using two kinds of attacks: OAA and AAA, as described in Section 5.21 The aim is produce k- corrupted responses in the whole dataset. The prior coefficient Wo is generated by w + lu, where vector and is non-negative number is to 0.5 unless otherwise stated). En takes the form sL, where & takes different value for each method. All parameters are fixed in each experiment.
iid in (0, o2). We set
1
a random unit norm
Following the setting in [T], measured the performance of the regression coefficients by the standard L2 error: Tw w w2l... To judge whether the algorithm had converged, we used the termination criterion | w' 10 All results were averaged over 10 runs.
5.2 Corruption Methods
To demonstrate the efficiency of our proposed methods, we apply two different attacks to the dataset: OAA and AAA. The details of these two attacks are shown follows.
OAA: The set of corrupted points S is selected as a uniformly random k-sized subset of [n]]. and the corresponding response variables set as y; x² wn + b: + G, where b; are sampled from the uniform distribution U10,, 10] and the white noise ~ o²)
AAA: We use all information from the true data distribution to corrupt the data, and propose an adaptive data corruption algorithm (ADCA), This algorithm is quite similar to TRIP; full details of ADCA (Algorithm g) are given in Appendix B. ö is set to 0.1m for 1000, p = 200, and to 0.2n for 7 2000, p = 100.
Both TRIP and BRHT employ the prior Pw (w) while the variance Eg of Pw (w) in OAA is set to be four times higher than that in AAA, which means that the prior has higher weight in AAA. Other parameters for these two algorithms are fixed. The prior distribution (r(r) is set to the Gamma distribution unless otherwise stated.
We also design another leverage point attack (LPA) to test the robustness of our methods. More results can be seen in Appendix E.
5.3 Methods Comparison
Our methods are compared with three baselines: 1) CRR [I]] is an effective robust regression method in cases where there large numbers of random outliers; 2) Reweighted robust Bayesian regression (RRBR) [24!] allows us to judge whether our proposed methods are better than the original method; 3) Rob-ULA [3}] is an effective robust Bayesian inference method that approximately converges to the real posterior distribution in finite number of steps in the presence of outliers. The parameters of RRBR are the same as in the BRHT algorithm, except for the hard thresholding part. For more comparisons of other methods, please see Appendix E.

---

--101E.-SSLLAAA
-CRB
CHE BAN-TRA BISHET
TUR Baas DESIT
BEIITT
Carraption Radio
Corruntion Ratie
Carraption Radio
Cormuption Ratie (d)
(b)
(C)
Figure 1: Recovery of parameters with respect to the number of data points dimensionality d. and corruption ratio a. TRIP and BRHT are more robust under AAAs than CRR, and BRHT exhibits the best performance in all experiments. RRBR and uob- show some robustness, but offer slightly worse recovery in some experiments.
1860,0 2.t.:ddd-011--..0._... AAAAA
HIP Camtra Legesurman
-CHI BRITT
6d
Steradian stop
Suraaiin dlep
Corruption Ratlo
Priur Accuracy
(a)
(b)
(C)
(d)
Figure 2: (a), (b) show the convergence characteristics of TRIP and BRHT. Both algorithms exhibit an estimation bias as the price of adding a prior in OAA. but BRHT is more accurate and behaves significantly better in AAA, while TRIP stalls during the iterative process. (c), (d) show the conver- gence under different weights of the prior (r(r) and the coefficient prior Pw (w).
5.4 Recovery Properties of Coefficients and Uncorrupted Sets
CRR, RRBR, and ULA are excellent robust regression methods or Bayesian inference methods, but they all show their limitations in the face of different kinds of attacks. CRR achieves the best performance in the face of OAAs because is theoretically unbiased, but it collapses rapidly when facing AAAs, as shown in Figures Iice) and (ld). RRBR and Rob-ULA take priors into consider- ation, but RRBR cannot resist AAAs and Rob-ULA produces poor results under OAAs, as shown in Figures (Idd) TRIP produces good effect against AAAs, while BRHT is not only optimal against AAAs, but also displays a similar effect to CRR in the case of OAAs. This shows that BRHT the most robust algorithm among those compared in this experiment.
The TRIP and BRHT algorithms are compared in Figures ((a) and (bb). The TRIP method incor- porates much prior information in the those of BRHT and CRR as shown in Figure ((a)). However, under AAAs, the prior information of TRIP and BRHT is enhanced by a factor of four, as described in Section 5.2 BRHT converges under a weaker prior, while TRIP becomes trapped around a local optimum. This shows that BRHT only needs to integrate weak prior information the convergence properties under different weights of the prior (r(r) and coefficient prior pw (w). Figure (ic) shows that BRHT is not especially sensitive the weight prior (r( when this prior is relatively reliable. A log-normal distribution is the best choice when the prior Pw (w) is relatively close to the real parameters and Gamma distribution is more robust when the prior is imprecise, as shown in Figure (dd).
of OAAs, resulting in greater estimation error than
ensure convergence. Figures (ic) and (d illustrate
6 Conclusion
This paper has described a novel robust regression algorithm named TRIP that achieves strong results in terms of resisting AAAs. By adding a prior to the robust regression via hard thresholding, the recovery of coefficients is significantly improved. Another algorithm, named BRHT, was designed to improve the robustness of TRIP and reduce the estimation error through the use of Bayesian reweighting regression. We prove that both algorithms have strong theoretical guarantees and that the algorithms converge linearly under mild condition. Extensive experiments have illustrated that our algorithms outperform benchmark methods in terms of both robustness and efficiency.

---

There are several interesting future directions to extend current work. Firstly, in this article, we only consider the case when y is corrupted. One would consider using the prior information better deal with the problem where both y and X are corrupted. Secondly, it would be also interesting to further reduce the effect prior on the estimation to make it consistent.
10

---

Acknowledgments and Disclosure of Funding
This work partly supported by National Key Research and Development Program of China (2021 1000300 and (EAYF0010000))) National Center for Mathematics and Interdisciplinary Sciences, and Key Laboratory of Systems and Control of CAS.
References
[1] Kush Bhatia, Prateek Jain, Parameswaran Kamalaruban, and Purushottam Kar. Consistent robust regression. Advances in Neural Information Processing Systems, 30, 2017.
[2] Kush Bhatia, Prateek Jain, and Purushottam Kar. Robust regression via hard thresholding. Advances in Neural Information Processing Systems, 28, 2015.
[3] Kush Bhatia, Yi-An Ma, Anca D Dragan, Peter L Bartlett, and Michael Jordan. Bayesian robustness: A nonasymptotic viewpoint. arXiv preprint arXiv:1907.1 1199977.18266. 2019.
[4] Yeshwanth Cherapanamjeri, Efe Aras, Nilesh Tripuraneni, Michael I. Jordan, Nicolas Flam- marion, and Peter L. Bartlett. Optimal robust linear regression in nearly linear time. arXiv preprint arXiv:2007.083 17, 2020.
[5] Peter Congdon. Buyesian statistical modelling. John Wiley & Sons, 2007.
[6] Ilias Diakonikolas, Weihao Kong, and Alistair Stewart. Efficient algorithms and lower bounds for robust linear regression. In Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms,, pages 2745-2722 2754.. SIAM, 2019.
[7] Çalar Gülçehre and Yoshua Bengio. Knowledge matters: Importance of prior information for optimization. Journal of Machine Learning Research, 17(1):226-23 257,62257220 2016.
[8] Chao Huang, Dong Wang, and Nitesh V Chawla. Scalable uncertainty-awes truth discovery in big data social sensing applications for cyber-physical systems. IEEE Transactions on Big Data, 6(4):702-713 2017.
[9] Arun Jambulapati, Jerry Li, Tselil Schramm, and Kevin Tian. Robust regression revisited: Acceleration and improved estimation rates. In Advances Neural Information Processing Systems, volume 34, pages 4475-4488. Curran Associates, Inc., 2021.
[10] Sushrut Karmalkar and Eric Price. Compressed sensing with adversarial sparse noise via 11 regression. arXiv preprint arXiv:1809.08055 2018.
[11] Xi Li, Xiaoling Chen, Yousong Zhao, Jia Xu, Fengrui Chen, and Hui Li. Automatic intercali- bration of night-time light imagery using robust regression. Remote sensing letters, 4(1):45 54, 2013.
[12] Ming Lin, Xiaomin Song, Qi Qian, Hao Li, Liang Sun, Shenghuo Zhu, and Rong Jin. Robust Gaussian process regression for real-time high precision GPS signal enhancement. In Proceed- ings of the 25th ACM SIGKDD International Conference Knowledge Discovery Data Mining, pages 2838-2847, 2019.
[13] Vanda M Lourenço. Ana M Pires, and M Kirst. Robust linear regression methods in association studies. Bioinformatics, 27(6):815-8211 2011.
[14] Brian McWilliams, Gabriel Krummenacher, Mario Lucic, and Joachim M Buhmann, Fast and robust least squares estimation in corrupted linear models. Advances in Neural Information Processing Systems, 27, 2014.
[15] Ankit Pensia, Varun Jog, and Po-Ling Loh. Robust regression with covariate filtering: Heavy tails and adversarial contamination. arXiv preprint arXiv:2009. 12976, 2020.
[16] Scott Pesme and Nicolas Flammarion. Online robust regression via SGD on the I loss. In Advances in Neural Information Processing Systems, volume 33, pages 2540-2552. Curran Associates, Inc., 2020.

---

[17] Nicholas G Polson and James G Scott. Shrink globally, act locally: Sparse Bayesian regular- ization and prediction. Bayesian Statisties, 01055):01-533, 2010.
[18] Adarsh Prasad, Arun Sai Suggala, Sivaraman Balakrishnan, and Pradeep Ravikumar. Robust estimation via robust gradient estimation. arXiv preprint arXiv:/802.0644 2018.
[19] Christoph Studer, Patrick Kuppinger, Graeme Pope, and Helmut Boleskei. Recovery of sparsely corrupted signals. IEEE Transaction on Information Theory, 5855331153310, may 2012.
[20] Arun Sai Suggala, Kush Bhatia, Pradeep Ravikumar, and Prateek Jain. Adaptive hard thresh- olding for near-optimal consistent robust regression. In Proceedings of the Thirty-Second Con- ference on Learning Theory, volume 99 of Proceedings of Machine Learning Research, pages 1892-2892 PMLR, 25 28 Jun 2019.
[21] C Ming Wang, Dominic F Vecchia, Matt Young, and Nathan A Brilliant. Robust regression applied to optical-fiber dimensional quality control. Technometrics, 39(1):25-33, 1997.
[22] Chong Wang and David M Blei. general method for robust Bayesian modeling. Bayesian Analysis, 1164 1191, 2018.
[23] Donglin Wang, Don Hong, and Qiang Wu. Prediction of loan for mortgage data: Deep learning versus robust regression. Computational Economics, pages 1-14, 2022.
[24] Yixin Wang, Alp Kucukelbir, and David M Blei. Robust probabilistic modeling with Bayesian data reweighting. International Conference on Machine Learning, pages 3646-3655. PMLR, 2017.
[25] He Yan, Yong Qi, Qiaolin Ye, and Dong-Jun Yu. Robust least squares twin support vector regression with adaptive foa and pso for short-term traffic flow prediction. IEEE Transactions on Intelligent Transportation Systems, 2021.
[26] Abdelhak M Zoubir, Visa Koivunen, Esa Ollila, and Michael Muma. Robust statisties for signal processing. Cambridge University Press, 2018.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you descrihe the limitations of your work? [Yes] Section and
(c) Did you discuss any potential negative societal impacts of your work? [No] Our research concentrate on regression and no social risks are involved.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Appendix C.
(b) Did you include complete proofs of all theoretical results? [Yes] Appendix C.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or URLY? [No] The experi- ments are based on simulation and easy to reproduce under given parameters. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Section 5.1, 5.2 and Appendix D.
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [No] The random experiments will demonstrate the results.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No] The overall calculation scale is small.
4. If you using existing assets (e.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] In each time we first mention other methods.
(b) Did you mention the license of the assets? [No] We only reference other algorithms.
(c) Did you include any new assets either in the supplemental material or as a URL? [No] Our experiments don't 't need that.
(d) Did you discuss whether and how consent was obtained from people whose data you'te re using/curating? [N/A] We don't 'o use other's data.
(e) Did you discuss whether the data you using/curating contains personally identifi- able information or offensive content? [N/A] Our data are randomly generated and do not involve these situations.
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "yUZ71Qvw_5", "reviewer": "Reviewer_DbYe", "summary": "The paper studies the problem of robust linear regression and demonstrates two algorithms which improves the breakdown point for this algorithm under adversarial attacks compared to prior work.\n\nThe main contribution is the observation that, with some prior information, even if it is far from the truth, it is possible to improve the breakdown point and find a consistent estimator for the problem. Prior work was able to find consistent estimators only in the setting where the corruption vector was chosen oblivious to the measurements. \n\nThe authors consider the Adaptive Adversarial Attacks (AAA)  setting where the observations are given by $(X, y)$, where $X \\in \\mathbb{R}^{d \\times n}$ and $y \\in \\mathbb{R}^n$. The observations satisfy $y = Xw^* + b^* + \\epsilon$ where $w^*$ is the true regression coefficient, $\\epsilon$ is the dense noise and $b^*$ is an arbitrary $k^*$-sparse vector. The goal is to recover $(\\hat{w}, \\hat{S})$ where $\\hat{w}$ is close to $w^*$ and $\\hat{S}$ is an estimate of the uncorrupted sample set.  The authors assume that the variance of the noise $\\epsilon$ can be controlled by the algorithm designer and be set to $\\sigma$, or can be estimated independently to a high degree of accuracy. \n\nThe first algorithm (TRIP) assumes the prior on $w*$ is $\\mathcal N(w_0, \\Sigma_0)$ for some $w_0$ and $\\Sigma_0$ determined in advance. This essentially leads to the problem being transformed into a regularised least squares problem, which they solve by a hard-thresholding approach similar to [1]. The second algorithm is more complicated and assumes a prior also on the local weight assigned to specific samples and uses variational Bayesian expectation maximization to solve a reweighted probabilistic model for linear regression. \n", "questions": "1. It would be good to explicitly write down the breakdown point achieved in the case of a Gaussian. For instance [2] explicitly says that one lower bound for the breakdown point is 1/65. \n2. I would be curious to see how this algorithm performs with the following noise model from [3]: Samples $(x, y)$ where $x \\sim N(0, 1)$ and $y = 100 x$, where the adversary looks at the $\\epsilon$ fraction of the $x_i$ such that $|x_i|$ is maximized, and then sets the corresponding $y$ to 0. The plot of the error $\\| \\widehat w - w^* \\|$ vs $\\epsilon$ might shed light on the comparison between breakdown points in the case of a Gaussian. While I understand that the goal is to provide a consistent estimator, It might be clearer to see the performance of the algorithm and the breakdown point for an example with no additive error. \n3. Could the authors please explain for my understanding, intuitively, why the existence of the prior allows one to solve the problem in the harder adversarial setting? Somehow this has still escaped me. \n\nI will reconsider my score depending on the answers to these questions. \n\n-----\n\nUpdate: The authors have addressed all questions raised, and I have updated the score to reflect my new assessment. I still have some concern about the theoretical breakdown point, but the algorithms seem to perform well on noise models which previous algorithms fail on. ", "limitations": "This is a theoretical result with limited societal impact. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\nIt is qualitatively interesting that bayesian updates can lead to consistent estimators in the AAA setting. I like the result. \n\n\nWeaknesses: \n\n1. I feel that the authors need to place the result in the context of the surrounding literature a little better. This paper considers the model where only the labels are corrupted (i.e. only $y$) while the result by Diakonikolas et al. [6] (mentioned on line 53) is for the setting where both the $X$ as well as $y$ values might be corrupted. \n\n2. Additionally, further comparison with [2] would be helpful, since the goal of this paper seems to be to achieve a good breakdown point for the AAA setting, which I feel [2] also addresses, as opposed to [1] which only deals with the oblivious setting (with the additional goal of being a consistent estimator, i.e. if the dense additive noise is mean 0, the goal here is for the estimator to achieve error going to 0 as the number of samples tends to infinity). \n\n3. Another paper to cite might be https://arxiv.org/abs/1809.08055, which attempts to solve the problem using L1-regression but it appears this is not a consistent estimator. \n\n4. I found the paper a little hard to understand. Clearer motivation and a description of the algorithm would have been helpful. I would also have liked there to be more intuition and a comparison of the difference in terms of the algorithms and updates from [1] and [2] that would help the reader better understand how the assumption of the prior helps. \n\n[1] Kush Bhatia, Prateek Jain, Parameswaran Kamalaruban, and Purushottam Kar. Consistent robust regression. Advances in Neural Information Processing Systems, 30, 2017.\n\n[2] Kush Bhatia, Prateek Jain, and Purushottam Kar. Robust regression via hard thresholding. Advances in Neural Information Processing Systems, 28, 2015.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "obPS0PGa756", "reviewer": "Reviewer_AqFQ", "summary": "The paper works on robust least-squares regression of the form $\\mathbf{y} = X^{T}\\mathbf{w}^* + \\mathbf{b}^* + \\boldsymbol{\\epsilon}$, where $\\mathbf{b}^*$ represents a $k$-sparse adversarial perturbation. Specifically, it proposes a Bayesian extension of the Hard Thresholding method previously designed against oblivious adversarial attacks (OAAs), to make it resistant against (a stronger) adaptive adversarial attacks (AAAs), that assumes an adversary having an access to $X$, $\\mathbf{w}^*$ and $\\boldsymbol{\\epsilon}$ before attacks. Two methods are proposed, namely TRIP and BRHT, that incorporate a Gaussian prior on weights $w$ and additionally on exponential weighting $r$, respectively. The paper provides both theoretical convergence analysis and empirical results showing that the proposed methods are more robust under AAAs while also improving under the prior setup of OAAs. ", "questions": "- Section 3.1: Is the proposed hard-thresholding based method actually induced from assuming Gaussian weight prior, or they are just orthogonal, i.e., TRIP works without the prior? More explanation on the relationship between the prior assumption and TRIP may help.\n- For TRIP, I feel it is a bit unclear why one should assume the Gaussian prior for robustness against AAAs, but perhaps not for others? If not, the paper could empirically explore the effect of using different priors on weights.\n- Line 115: More discussions could be made on how mild the assumed SSC and SSS conditions in both theoretical and empirical senses?", "limitations": "The paper does not include discussions on potential negative societal impact.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- The paper is clearly-written, e.g., it clearly presents the problem formulation and methodologies.\n- The proposed methods are simple, and well-motivated.\n- The paper conducts empirical analysis to support the claims and reports improved results. In the experiments, the paper designs an adaptive attack scheme ADCA in a similar manner to TRIP but in the opposite direction in optimization.\n\n**Weaknesses** \n\n- Given that AAA is assumed to be an adversary, a more diverse attack methods could be considered - the current attack method of ADCA is designed to be quite close to TRIP, and it is likely that TRIP performs better than others under ADCA because it is just trained as an opposite of ADCA and may not generalize under other ways of attacks.\n- Although it is still novel to me that the paper aims for adversarial robustness, Bayesian approaches on weights and loss coefficients have been quite known approaches in general so some readers may concern on the technical novelty on the method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "X5fZxRfRbUI", "reviewer": "Reviewer_eKkz", "summary": "The authors study the robust least-squares regression (RLSR). The main contribution of this paper is to propose an algorithm that achieves strong results in terms of resisting adaptive adversarial attacks. A theoretical convergence analysis is provided. Extensive experiments have illustrated that their algorithms outperform SOTA methods in terms of both robustness and efficiency.", "questions": "It is better to conduct more experiments in more complicated settings.", "limitations": "Yes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 5, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: The related works are adequately cited. The novelty of this paper is high. The results on robust least-squares regression in this paper, will certainly help us have a better understating of adversarial attacks and defenses from a theoretical way. I have checked the technique parts and find that the proofs are solid. I think this is a significant contribution to machine learning immunity. \n\nQuality: This paper is technically sound.\n\nClarity: This paper is clearly written and well organized. I find it easy to follow.\n\nSignificance: I think the results in this paper is significant, as explained above.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept