Paper: An Efficient Tester-Learner for Halfspaces (ICLR-2024)

Reviewer comment (Theory):
I believe the results easily extend to non-homogeneous halfspaces where there is a constant offset term?

Author response:
We believe that tester-learners which handle linear classifiers with non-zero offset terms may be significantly different from ours, given the fact that in standard distribution-specific agnostic learning the techniques used to handle halfspaces with non-zero offset terms are quite different from the techniques used here. See [Diakonikolas, Kontonis, Tzamos, Zarifis, ‘22] and [Diakonikolas, Kane, Stewart 2018] for further detail.

Reviewer comment (Writing):
I believe only the tester T3 uses the labels to check the fooling and T1 and T2 does not in Algo 1? Small typo: the Run T2 step has $\sigma$ missing in $B'_W(\sigma)$.

Author response:
The testers $T_1, T_2$ and $T_3$ do not use labels internally. However, testers $T_2$ and $T_3$ receive $\mathbf w$ as an input and $\mathbf w$ is computed using labelled examples (see Algorithm 1). Therefore, the tests we perform are indeed label-dependent, but the testers $T_1,T_2,T_3$ do not need to receive labelled examples per se. Additionally, thank you for pointing out the typo.

Reviewer comment (Theory):
I am confused why the algorithm needs two $T_3$ testers with different accuracies, $\sigma/6$ and $\sigma/2$?

Author response:
Running tester $T_3$ with a given parameter $\sigma$ does not necessarily guarantee this tester accepts also for bigger parameter values $\sigma’$. Intuitively, this is because $\sigma$ denotes the width of a band around a given halfspace that the tester focuses on. Our analysis requires that the distribution is well-behaved for two different values of the band size $\sigma/2$ and $\sigma/6$, and therefore we run the tester $T_3$ for these two values separately.

Reviewer comment (Theory):
I am not very familiar with tester-learner models. Are there lower bounds on learning gaussian halfspaces under massart/agnostic noise for tester-learner algorithms? Are tester-learner algorithms SQ algorithms?

Author response:
Since every testable learning algorithm will also satisfy the requirements for a distribution-specific agnostic learning, known hardness results for distribution-specific agnostic learning (such as [DK '22] and [DKMR '22]) also imply hardness of testable learning. We note that our tester-learner algorithms can be formulated in the SQ framework.

Reviewer comment (Reproducibility):
In algorithm 1, what exactly is the function class $\mathcal{F}_{w'}$? How to choose the weights that are orthogonal to $w'$?

Author response:
The algorithm 1 does not compute function class $\mathcal F_{w}$, we only mention it as a comment for the reader to better understand what the various testers $T_1$, $T_2$ and $T_3$ accomplish. In particular, the class $\mathcal F_{w}$ is the class of functions that the marginal distribution (conditioned on the strip) has to fool in order to ensure soundness. For example, one function in $\mathcal F_{w}$ is $f$ such that $f(x) = 1$ for any $x\in A_2$ ($A_2$ as in Figure 1) and $f(x)=-1$ for any $x$ in the strip ($|x\cdot w| \le \sigma$), but $x$ not in $A_2$. Note that $f$ can be defined as an intersection of two halfspaces (orthogonal to $w$), since we only care about the values within the strip.