Paper: Oracle Efficient Algorithms for Groupwise Regret (ICLR-2024)

Reviewer comment (Experiments):
It is not obvious that you should have an "always active" subsequence in order to perform well empirically. It would be good if it is reported that the algorithm does not work so well if this is not present, so others people that want to implement this can take this fact into account.

Author response:
Thank you for the review and the experimental suggestion. On your suggestion, we have added to the revision new experiments in Appendix F, in which we see that including/excluding the "always on'' expert has a marginal effect on the regret of our algorithm: both are much better than the baseline.  

We made the choice to include the "always on'' expert on theoretical grounds. Doing so provides a regret bound over the entire sequence, which does not follow generically from having regret bounds for subsequences. If there are subsequences that partition the entire sequence, then one gets a regret bound on the entire sequence that follows from summing up the regret bounds on the partition but this is wasteful. As an extreme example, consider the case in which each time step corresponds to a separate group. In this case group-wise regret bounds alone do not give any non-trivial regret guarantee over the full sequence. If on the other hand the subsequences do not partition the space, then there is not necessarily a regret bound on the entire sequence without including the "always on'' expert at all. In our experiments, there are indeed (large) groups that partition the sequence, which is consistent with the experimental results showing that the "always on'' expert is not needed in these instances.

Reviewer comment (Theory):
Theorem 1 and Theorem 3 say essentially the same. It's redundant.

Author response:
We have fixed the minor errors you have pointed out and removed our informal Theorem 1, to help focus the reader on the formal statements of our results.

Reviewer comment (Writing):
"low order regret terms" -> "low-order regret terms".

Author response:
We have fixed the minor errors you have pointed out and removed our informal Theorem 1, to help focus the reader on the formal statements of our results.

Reviewer comment (Writing):
Thm 1 (informal): contains "of of".

Author response:
We have fixed the minor errors you have pointed out and removed our informal Theorem 1, to help focus the reader on the formal statements of our results.

Reviewer comment (Writing):
"to each Individual" remove capitalization.

Author response:
We have fixed the minor errors you have pointed out and removed our informal Theorem 1, to help focus the reader on the formal statements of our results.

Reviewer comment (Theory):
The paper is not presented in a fully self-contained and rigorous manner. For instance, the proposed algorithm uses AdaNormalHedge as a black box, but the guarantee of such an algorithm is never formally described. Although there is a description in Appendix B, there is no proposition and proper citation for this. Similarly, when citing external regret minimization algorithms for applications, the formal quantifiers and guarantees for those algorithms are not provided.

Author response:
Thank you for the thoughtful review and writing suggestions. We will take this into account as we revise the paper and aim to improve the exposition. In the short term (in the revision we have uploaded) we have added a more formal section on sleeping experts and AdaNormalHedge, as well as a formal statement of its regret guarantees in Appendix B.  

We kept the introduction informal for the reason you guessed: to make the goals of the paper accessible to a wider audience --- but we understand your concern that this makes the paper harder to approach for a more technical reader. At the moment, the precise definitions of \'\'no-regret'' and \'\'best-in-hindsight'', are provided in Section 2. We will consider moving them to the introduction. You are right that our results hold for general loss and not just squared loss: we use squared loss as an exemplar both because it is very common, and because our experimental results are for (squared loss) linear regression. We will clarify that our framework is not limited to squared loss. 

To answer your questions:  \'\'diminishing external regret'' refers to the regret scaling sub-linearly in the length of the sequence, so that the average regret tends to $0$ with the length of the sequence.  We have added a note to that effect in the introduction. Computationally efficient here is defined as oracle-efficient with respect to the model class $H$ and polynomial run time with respect to the other parameters of the problem ($|G|$, $d$, $n$). This means that our algorithms should run at most a polynomial number of the online learning ``oracles'' and have additional over-head that is polynomial in the problem parameters. We will clarify. 

Thanks for pointing out the issue with the first inequality --- there was indeed a typo, which we have fixed now. On your suggestion, we have expanded on the proof of Theorem 2, and in particular have made an effort to be explicit when it comes to the coins over which the expectations are taken.

Reviewer comment (Writing):
The introduction is written in a very informal way. While this may be intended to accommodate a broader readership, it adds to confusion. For example, when defining the notion of diminishing group-wise regret, it would be more helpful to include the actual mathematical definition of “squared error” and “best model in H on that sequence.” Additionally, it is unclear why the notion is limited to squared error and not extended to general loss functions.

Author response:
Thank you for the thoughtful review and writing suggestions. We will take this into account as we revise the paper and aim to improve the exposition. In the short term (in the revision we have uploaded) we have added a more formal section on sleeping experts and AdaNormalHedge, as well as a formal statement of its regret guarantees in Appendix B.  

We kept the introduction informal for the reason you guessed: to make the goals of the paper accessible to a wider audience --- but we understand your concern that this makes the paper harder to approach for a more technical reader. At the moment, the precise definitions of \'\'no-regret'' and \'\'best-in-hindsight'', are provided in Section 2. We will consider moving them to the introduction. You are right that our results hold for general loss and not just squared loss: we use squared loss as an exemplar both because it is very common, and because our experimental results are for (squared loss) linear regression. We will clarify that our framework is not limited to squared loss. 

To answer your questions:  \'\'diminishing external regret'' refers to the regret scaling sub-linearly in the length of the sequence, so that the average regret tends to $0$ with the length of the sequence.  We have added a note to that effect in the introduction. Computationally efficient here is defined as oracle-efficient with respect to the model class $H$ and polynomial run time with respect to the other parameters of the problem ($|G|$, $d$, $n$). This means that our algorithms should run at most a polynomial number of the online learning ``oracles'' and have additional over-head that is polynomial in the problem parameters. We will clarify. 

Thanks for pointing out the issue with the first inequality --- there was indeed a typo, which we have fixed now. On your suggestion, we have expanded on the proof of Theorem 2, and in particular have made an effort to be explicit when it comes to the coins over which the expectations are taken.

Reviewer comment (Theory):
The statement of Theorem 1 uses the notion of “computationally efficient” without defining it. The phrase “best model on hindsight” is also used informally and should be properly defined (overall vs. on group-wise sequences) with the proper quantifiers.

Author response:
Thank you for the thoughtful review and writing suggestions. We will take this into account as we revise the paper and aim to improve the exposition. In the short term (in the revision we have uploaded) we have added a more formal section on sleeping experts and AdaNormalHedge, as well as a formal statement of its regret guarantees in Appendix B.  

We kept the introduction informal for the reason you guessed: to make the goals of the paper accessible to a wider audience --- but we understand your concern that this makes the paper harder to approach for a more technical reader. At the moment, the precise definitions of \'\'no-regret'' and \'\'best-in-hindsight'', are provided in Section 2. We will consider moving them to the introduction. You are right that our results hold for general loss and not just squared loss: we use squared loss as an exemplar both because it is very common, and because our experimental results are for (squared loss) linear regression. We will clarify that our framework is not limited to squared loss. 

To answer your questions:  \'\'diminishing external regret'' refers to the regret scaling sub-linearly in the length of the sequence, so that the average regret tends to $0$ with the length of the sequence.  We have added a note to that effect in the introduction. Computationally efficient here is defined as oracle-efficient with respect to the model class $H$ and polynomial run time with respect to the other parameters of the problem ($|G|$, $d$, $n$). This means that our algorithms should run at most a polynomial number of the online learning ``oracles'' and have additional over-head that is polynomial in the problem parameters. We will clarify. 

Thanks for pointing out the issue with the first inequality --- there was indeed a typo, which we have fixed now. On your suggestion, we have expanded on the proof of Theorem 2, and in particular have made an effort to be explicit when it comes to the coins over which the expectations are taken.

Reviewer comment (Theory):
In Appendix B, the usage of expectation notation $\mathbb{E}[]$ is confusing. The derivation critically depends on controlling which variables the expectation is taken over, so the expectation notation should have a subscript explicitly stating the source of randomness. Furthermore, the distinction between $p_{t}^{I}$ and $z_{t}^{I}$ is not rigorous enough. If $p_{t}^{I}$ is a random variable with realizations denoted as $z_{t}^{I}$, then in the first inequality, the term should be $\mathbb{E}[\sum_{t} I(t) \ell(p_{t}^{I}, y_{t})]$. This section overall has room for improvement.

Author response:
Thank you for the thoughtful review and writing suggestions. We will take this into account as we revise the paper and aim to improve the exposition. In the short term (in the revision we have uploaded) we have added a more formal section on sleeping experts and AdaNormalHedge, as well as a formal statement of its regret guarantees in Appendix B.  

We kept the introduction informal for the reason you guessed: to make the goals of the paper accessible to a wider audience --- but we understand your concern that this makes the paper harder to approach for a more technical reader. At the moment, the precise definitions of \'\'no-regret'' and \'\'best-in-hindsight'', are provided in Section 2. We will consider moving them to the introduction. You are right that our results hold for general loss and not just squared loss: we use squared loss as an exemplar both because it is very common, and because our experimental results are for (squared loss) linear regression. We will clarify that our framework is not limited to squared loss. 

To answer your questions:  \'\'diminishing external regret'' refers to the regret scaling sub-linearly in the length of the sequence, so that the average regret tends to $0$ with the length of the sequence.  We have added a note to that effect in the introduction. Computationally efficient here is defined as oracle-efficient with respect to the model class $H$ and polynomial run time with respect to the other parameters of the problem ($|G|$, $d$, $n$). This means that our algorithms should run at most a polynomial number of the online learning ``oracles'' and have additional over-head that is polynomial in the problem parameters. We will clarify. 

Thanks for pointing out the issue with the first inequality --- there was indeed a typo, which we have fixed now. On your suggestion, we have expanded on the proof of Theorem 2, and in particular have made an effort to be explicit when it comes to the coins over which the expectations are taken.

Reviewer comment (Theory):
Is the notion of computationally efficient in Theorem 1 defined as polynomial time in $T$, $d$, and $n$ (or some other input-related size)? The understanding is that it is reasonable to assume $|G|$ is polynomial in the input size, but $|H|$ is usually quite large. Therefore, having an algorithm that does not scale with $|H|$ implies polynomial-time efficiency.

Author response:
Thank you for the thoughtful review and writing suggestions. We will take this into account as we revise the paper and aim to improve the exposition. In the short term (in the revision we have uploaded) we have added a more formal section on sleeping experts and AdaNormalHedge, as well as a formal statement of its regret guarantees in Appendix B.  

We kept the introduction informal for the reason you guessed: to make the goals of the paper accessible to a wider audience --- but we understand your concern that this makes the paper harder to approach for a more technical reader. At the moment, the precise definitions of \'\'no-regret'' and \'\'best-in-hindsight'', are provided in Section 2. We will consider moving them to the introduction. You are right that our results hold for general loss and not just squared loss: we use squared loss as an exemplar both because it is very common, and because our experimental results are for (squared loss) linear regression. We will clarify that our framework is not limited to squared loss. 

To answer your questions:  \'\'diminishing external regret'' refers to the regret scaling sub-linearly in the length of the sequence, so that the average regret tends to $0$ with the length of the sequence.  We have added a note to that effect in the introduction. Computationally efficient here is defined as oracle-efficient with respect to the model class $H$ and polynomial run time with respect to the other parameters of the problem ($|G|$, $d$, $n$). This means that our algorithms should run at most a polynomial number of the online learning ``oracles'' and have additional over-head that is polynomial in the problem parameters. We will clarify. 

Thanks for pointing out the issue with the first inequality --- there was indeed a typo, which we have fixed now. On your suggestion, we have expanded on the proof of Theorem 2, and in particular have made an effort to be explicit when it comes to the coins over which the expectations are taken.

Reviewer comment (Theory):
The term “diminishing/vanishing regret” in Theorem 2 is unclear. The term $\sqrt{T_{T} \log(|G|)}$ is not $o(1)$ itself. Is there an implicit lower bound being enforced on $\alpha_{I}$?

Author response:
Thank you for the thoughtful review and writing suggestions. We will take this into account as we revise the paper and aim to improve the exposition. In the short term (in the revision we have uploaded) we have added a more formal section on sleeping experts and AdaNormalHedge, as well as a formal statement of its regret guarantees in Appendix B.  

We kept the introduction informal for the reason you guessed: to make the goals of the paper accessible to a wider audience --- but we understand your concern that this makes the paper harder to approach for a more technical reader. At the moment, the precise definitions of \'\'no-regret'' and \'\'best-in-hindsight'', are provided in Section 2. We will consider moving them to the introduction. You are right that our results hold for general loss and not just squared loss: we use squared loss as an exemplar both because it is very common, and because our experimental results are for (squared loss) linear regression. We will clarify that our framework is not limited to squared loss. 

To answer your questions:  \'\'diminishing external regret'' refers to the regret scaling sub-linearly in the length of the sequence, so that the average regret tends to $0$ with the length of the sequence.  We have added a note to that effect in the introduction. Computationally efficient here is defined as oracle-efficient with respect to the model class $H$ and polynomial run time with respect to the other parameters of the problem ($|G|$, $d$, $n$). This means that our algorithms should run at most a polynomial number of the online learning ``oracles'' and have additional over-head that is polynomial in the problem parameters. We will clarify. 

Thanks for pointing out the issue with the first inequality --- there was indeed a typo, which we have fixed now. On your suggestion, we have expanded on the proof of Theorem 2, and in particular have made an effort to be explicit when it comes to the coins over which the expectations are taken.

Reviewer comment (Experiments):
I believe that including an experimental comparison with the Blum & Lykouris (2019) approach would better justify the superiority of the new method.

Author response:
Thanks for the suggestion, which we take seriously. We considered whether we should attempt to implement and compare to [BL19], but opted against it. The difficulty with [BL19] is that it requires a small finite class of functions, which is a property that is not realized by any reasonable "hypothesis class''. For example, we focus on linear regression (perhaps the simplest widely used hypothesis class) --- but this is a continuous model class. To run [BL19], it would be necessary to discretize the set of linear predictors: if we discretized each coordinate to granularity $\epsilon$, in $d$ dimensions this would require $(1/\epsilon)^d$ hypotheses. This would be infeasible for anything beyond a very small number of dimensions, and we don't think it would be enlightening. This is  why [BL19] has not been implemented in prior work: indeed, we view the main contribution of our paper as making the approach initiated by [BL19] practical. 

References:

[BL19] Avrim Blum and Thodoris Lykouris. Advancing subgroup fairness via sleeping experts. In Innovations in Theoretical Computer Science Conference (ITCS), volume 11, 2020.

Reviewer comment (Experiments):
Is it possible to conduct such a comparison using a smaller model class?

Author response:
Thanks for the suggestion, which we take seriously. We considered whether we should attempt to implement and compare to [BL19], but opted against it. The difficulty with [BL19] is that it requires a small finite class of functions, which is a property that is not realized by any reasonable "hypothesis class''. For example, we focus on linear regression (perhaps the simplest widely used hypothesis class) --- but this is a continuous model class. To run [BL19], it would be necessary to discretize the set of linear predictors: if we discretized each coordinate to granularity $\epsilon$, in $d$ dimensions this would require $(1/\epsilon)^d$ hypotheses. This would be infeasible for anything beyond a very small number of dimensions, and we don't think it would be enlightening. This is  why [BL19] has not been implemented in prior work: indeed, we view the main contribution of our paper as making the approach initiated by [BL19] practical. 

References:

[BL19] Avrim Blum and Thodoris Lykouris. Advancing subgroup fairness via sleeping experts. In Innovations in Theoretical Computer Science Conference (ITCS), volume 11, 2020.