Paper: Unified Projection-Free Algorithms for Adversarial DR-Submodular Optimization (ICLR-2024)

Reviewer comment (Novelty):
Could you elaborate on the technical challenges of this paper? The proposed algorithms seem to combine existing techniques without introducing novel technical tools.

Author response:
We note that technical novelty is subjective, and what one considers innovative or novel can vary based on individual perspectives within the research community. Some researchers may emphasize the introduction of entirely new tools, while others might appreciate creative combinations of existing ideas or the application of known methods in a new context. In that sense, many open problems have been resolved by creative combinations of existing ideas, and thus is perceived to be novel by the research community.

Consider for example the Chernoff bound, which was first described in a 1952 paper, decades after Markov and Chebyshev.
It provides a new insight which allows for sharper concentration bounds compared to Markov's inequality.
We consider the idea of Chernoff bound to be a technical novelty, even though the result is an almost immediate corollary of Markov's inequality.
Our approach of unified algorithm and analysis extends multiple ideas beyond the settings they were originally designed for, and gives new insights to the area. These insights allowed us to improve the state of the art in almost all cases.

Oftentimes the difference between "a novel combination of existing ideas" and "a new idea" is in having a new name.
Many times in machine learning, we have seen that old ideas have been used with a new name and considered novel.
However, we have tried to give credit to previous works as much as possible. In fact, if we are being precise, 9 out of the 16 previous algorithms mentioned in Table 1 either require stronger assumptions to work (See Appendix C on the boundedness of stochastic oracles, note that we have updated our paper) or have proved less than we have attributed to them by putting them in that table (See Appendix C for details).

In regards to the novelty of (ii), we note that realizing that a tool is not needed is by itself a novelty.
For example, consider the idea of Generative Pre-Trained (GPT) models.
GPT models were introduced in (Improving Language Understanding by Generative Pre-Training, Alec Radford et al., 2018) using only the decoder half of the Transformer model (Attention Is All You Need, Vaswani et al., 2017). They demonstrated that the attention mechanisms in the decoder layers enable the model to capture dependencies and context over long ranges, contributing to its effectiveness in various language-related applications.

We also note that the unification of multiple scenarios often involves recognizing connections and similarities between seemingly unrelated ideas. 
Unification is considered a novelty because it reflects the discovery of hidden connections, the creation of more general frameworks, and the ability to simplify and illuminate complex landscapes.

Finally, we note that (Chen et al., 2018b) combined the idea of meta-actions (Streeter et al., 2008) and the update rule of (Bian et al., 2017b) to obtain Meta-Frank-Wolfe.
(Chen et al., 2018a) combined the idea of (Chen et al., 2018b) and the momentum idea described in (Mokhtari et al., 2020).
(Mualem et al., 2023) combined the update rule of (Du et al., 2022) and the Meta-Frank-Wolfe idea of (Chen et al., 2018b) to obtain their algorithms.
(Zhang et al., 2023) combined the idea of (Chen et al., 2018b) and (Zhang et al., 2019) and a slightly modified version of the update rule in (Bian et al., 2017a) to obtain their results. 
We respectfully mention that based on the reviewer's comment, it feels that the reviewer is saying none of these results have any technical novelty since most of the tools were in existing literature, and should not have been accepted to top-tier venues. 
We believe that they are novel because they describe approaches to use known tools to obtain new results, and those combinations of ideas have not been thought of before.

Reviewer comment (Novelty):
This paper proposes two unified projection-free Frank-Wolfe type algorithms for the full-information and bandit settings. Could you provide the reasons why previous algorithms couldn't achieve a unified framework?

Author response:
**Q2**: We cannot speak for authors of other papers on why they did not propose a unified framework. 
Instead, we partly address a related question -- in what way is this work unified and what more work would some authors have needed to do in order to unify their separately described and analyzed algorithms.
In general, that would depend on how one defined the notion of "unification".
There are several different forms of unification appearing in our work:

**(a)** Unification of value/gradient oracle settings:  
Algorithm 1 is the first algorithm for adversarial DR-submodular maximization with full information feedback given a value oracle.
Except for (Chen et al., 2018b) which uses a projection-based method, Algorithm 2 is the first algorithm for adversarial DR-submodular maximization with semi-bandit feedback given a gradient oracle.
This is arguably the simplest form of unification present in our work.

Reviewer comment (Theory):
Compared with Zhang et al. (2023) for non-monotone functions, the proposed algorithms achieve only a slight improvement in terms of $\alpha$-regret guarantee. Can you explain the reasons for this, and the differences between two algorithms?

Author response:
**Q3**: We would like to gently push back on the statement "the proposed algorithms achieve only a slight improvement".
Note that we are aiming for sublinear regret, i.e., $o(T)$, and we do not expect the regret to be lower than $O(T^{1/2})$. Hence the whole range for improvement is from $O(T)$ to $O(T^{1/2})$.
(Zhang et al., 2023) proposed 3 algorithms: Meta-MFW, Mono-MFW, and Bandit-MFW.
Meta-MFW obtains a regret of $O(T^{1-\beta/3})$ with $T^{\beta}$ gradient queries. We obtain $O(T^{2/3 - \beta/3})$ which is an improvement of order $O(T^{1/3})$.
Mono-MFW obtains a regret of $O(T^{4/5})$ given a single gradient query. We obtain $O(T^{2/3})$ in the same setting which is an improvement of order $O(T^{2/15})$.
Finally, Bandit-MFW obtains a regret of $O(T^{8/9})$ given bandit feedback. We obtain a better bound of $O(T^{5/6})$ in a more general setting, i.e., noisy bandit feedback, which is an improvement of order $O(T^{1/18})$. 
Please see our responses to Q1-(i) and Q1-(ii) regarding the differences in algorithm design.

Reviewer comment (Novelty):
Could the authors provide a discussion on the technical contributions of this paper regarding online adversarial continuous DR-submodular optimization?

Author response:
(The fact that their algorithm obtains $O(T^{1-\beta})$ regret for $T^{\beta}$ queries per function follows from the first part of Theorem 4.1 together with the last sentence of Section 4 and the first paragraph of the proof Theorem 3.1 in their paper.)
While they state their algorithm and proof for an arbitrary number of queries per function, they have no analysis or discussion on query complexity, nor on the possibility of extending that approach to other settings.
To the best of our knowledge, our paper is the first to realize that removing momentum could result in improved regret bound for a fixed query complexity.

We prove that for online DR-submodular optimization, in all cases, even when the feedback is stochastic, using meta-actions alone (i.e., without momentum) is *sufficient* to obtain sublinear $\alpha$-regret bounds and using momentum decreases the performance.
We extend this analysis to Mono-Frank-Wolfe, where we improve the state of the art by $O(T^{2/15})$.
If the idea of random permutations is used as we have described in (i), together with momentum, the regret will likely be bounded from above by $O(T^{4/5 - \beta/5})$.
By removing momentum, we achieve $O(T^{2/3 - \beta/3})$ regret, i.e. a likely improvement of $O(T^{2(1 - \beta)/15})$ over the result one would obtain by using (i) with momentum.

For **(iii)**, we note that many papers and results add more complexity to the field of research, but our result tries to move in the opposite direction. Algorithm 1 (for full-information feedback) generalizes, simplifies and improves 8 previous algorithms, while Algorithm 2 (for semi-bandit feedback) does this for 3 previous algorithms.
We developed a unified analysis which is more modular and simpler than analyses in prior works.
This modularity allows us to *(a)* obtain better insight for how elements of the algorithm design and problem setting interact in bringing about the $\alpha$ approximation ratio separately from the $\alpha$-regret bounds; and as consequence, results in *(b)* proof for regret guarantees for 16 algorithms in shorter space than other papers proved regret guarantees for just one or two algorithms.

Regarding *(iii).(a)* on how the modularity of the analysis yields clearer insight, 
in Appendix E we show an essential lemma (Lemma 8) that makes explicit why the update rules and the step sizes should be the way they are. 
Lemma 8 only considers a single DR-submodular function and captures how certain update rules result in certain approximation coefficients.
In fact, besides the two algorithms in this paper, Lemma 8 may be directly used in the proof of at least 18 algorithms in the literature as the part of the proof which derives the approximation coefficients: (Chen et al., 2018b) (Chen et al., 2018a) (Zhang et al., 2019) (Niazadeh et al., 2021) (Thang et al., 2021) (Zhang et al., 2023) (Mualem et al., 2023) (Bian et al., 2017b) (Mokhtari et al,. 2020) (Bian et al., 2017a) (Du et al., 2022) (Pedramfar et al., 2023)

Regarding *(iii).(b)*, as an example of how much the modularity of our analysis leads to brevity, in the paper (Zhang et al., 2019), where the idea of random permutations is first used for DR-submodular maximization with full-information single sample gradient $T^\beta=1$ feedback, the proof of their result for the full-information setting 
is 8.5 pages (Appendix B in their paper).  
That proof only covers the  (simpler) case of monotone functions over convex sets containing the origin, with  a single evaluation of gradient oracle.
They use another 9 pages to prove their result for monotone functions over downward closed convex sets in the bandit setting.
(We are only considering the proofs that involve the ideas of meta-actions and random permutations, not general statements about convex sets or DR-submodular functions.)
In contrast, the only part of our proofs that deals explicitly with the online setting are in Appendix F and G, a total of 4 pages, which, together with Lemma 9, prove the regret bounds for 16 problem variations listed in Table 1.