SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Mohammad Pedramfar
mpedramf@purdue.edu
Christopher John Quinn
cjquinn@iastate.edu
vaneet@purdue.edu
This paper presents a unified approach for maximizing continuous DR-submodular functions that encompasses a range of settings and oracle access types. Our approach includes a Frank-Wolfe type offline algorithm for both monotone and non-monotone functions, with different restrictions on the general convex set. We consider settings where the oracle provides access to either the gradient of the function or only the function value, and where the oracle access is either deterministic or stochastic. We determine the number of required oracle accesses in all cases. Our approach gives new/improved results for nine out of the sixteen considered cases, avoids computationally expensive projections in three cases, with the proposed framework matching performance of state-of-the-art approaches in the remaining four cases. Notably, our approach for the stochastic function value-based oracle enables the first regret bounds with bandit feedback for stochastic DR-submodular functions.
The problem of optimizing DR-submodular functions over a convex set has attracted considerable interest in both the machine learning and theoretical computer science communities [2, 5, 16, 26]. This is due to its many practical applications in modeling real-world problems, such as influence/revenue maximization, facility location, and non-convex/non-concave quadratic programming [3, 9, 18, 15, 20]. as well as more recently identified applications like serving heterogeneous learners under networking constraints [20] and joint optimization of routing and caching in networks [21].
Numerous studies investigated developing approximation algorithms for constrained DR-submodular maximization, utilizing a variety of algorithms and proof analysis techniques. These studies have addressed both monotone and non-monotone functions and considered various types of constraints on the feasible region. The studies have also considered different types of oracles—gradient oracles and value oracles, where the oracles could be exact (deterministic) or stochastic. Lastly, for some of the aforementioned offline problem settings, some studies have also considered analogous online optimization problem settings as well, where performance is measured in regret over a horizon. This paper aims to unify the disparate offline problems under a single framework by providing a comprehensive algorithm and analysis approach that covers a broad range of setups. By providing a unified framework, this paper presents novel results for several cases where previous research was either limited or non-existent, both for offline optimization problems and extensions to related stochastic online optimization problems.
This paper presents a Frank-Wolfe based meta-algorithm for (offline) constrained DR-submodular maximization where we could only query within the constraint set, with sixteen variants for sixteen problem settings. The algorithm is designed to handle settings where (i) the function is monotone
This work was supported in part by the National Science Foundation under grants CCF-2149588 and CCF-2149617, and Cisco, Inc.
Table 1: Offline DR-submodular optimization results.
F	Set	Oracle	Setting	Reference	Appx.	Complexity
Monotone	0 ∈ K	∇F	det.	[4], (*)	1 - 1/e	O(1/ε)
stoch.	[23], (*) [32] ‡	1 - 1/e 1 - 1/e	O(1/ε3) O(1/ε2)
F	det.	This paper	1 - 1/e	O(1/ε3)
stoch.	This paper	1 - 1/e	O(1/ε5)
general†	∇F	det.	[16] ‡ This paper	1/2 1/2	O(1/ε)
O(1/ε)
stoch.	[16]‡ This paper	1/2 1/2	O(1/ε2)
O(1/ε3)
F	det.	This paper	1/2	O(1/ε3)
stoch.	This paper	1/2	O(1/ε5)
Non-Monotone	d.c.	∇F	det.	[3], (*)	1/e	O(1/ε)
stoch.	[23], (*)	1/e	O(1/ε3)
F	det.	This paper	1/e	O(1/ε3)
stoch.	This paper	1/e	O(1/ε5)
general	∇F	det.	[12] [11] [10], (*)	1-h 3√3 1-h 1/h 1/h 1/h	O(e√dL/ε) O(e√dL/ε) O(1/ε)
stoch.	This paper	1-h 1/h 1/h	O(1/ε3)
F	det.	This paper	1/h 1/h	O(1/ε3)
stoch.	This paper	1/h 1/h 1/h	O(1/ε5)
This table compares the different results for the number of oracle calls (complexity) within the feasible set for DR-submodular maximization. Shaded rows indicate problem settings for which our work has the first guarantees or beats the SOTA. The rows marked with a blue star (*) correspond to cases where Algorithm 2 generalizes the corresponding algorithm and therefore has the same performance. The different columns enumerate properties of the function, the convex feasible region (downward-closed, includes the origin, or general), and the oracle, as well as the approximation ratios and oracle complexity (the number of queries needed to achieve the stated approximation ratio with at most  $\epsilon > 0$  additive error). We have  $h := \min_{\mathbf{x} \in \mathcal{K}} \| x \|_{\infty}$ . (See Appendix B regarding [23] and [32]).  $\dagger$  when the oracle can be queried for any points in  $[0,1]^d$  (even outside the feasible region  $\mathcal{K}$ ), the problem of optimizing monotone DR-submodular functions over a general convex set simplifies — [4] and [23] achieve the same ratios and complexity bounds as listed above for  $0 \in \mathcal{K}$ ; [8] can achieve an approximation ratio of  $1 - 1/e$  with the  $O(1/\epsilon^3)$  and  $O(1/\epsilon^5)$  complexity for exact and stochastic value oracles respectively.  $\ddagger$  [16] and [32] use gradient ascent, requiring potentially computationally expensive projections.
or non-monotone, (ii) the feasible region is a downward-closed (d.c.) set (extended to include 0 for monotone functions) or a general convex set, (iii) gradient or value oracle access is available, and (iv) the oracle is exact or stochastic. Table 1 enumerates the cases and corresponding results on oracle complexity (further details are provided in Appendix A). We derive the first oracle complexity guarantees for nine cases, derive the oracle complexity in three cases where previous result had a computationally expensive projection step [32, 16] (and we obtain matching complexity in one of these), and obtain matching guarantees in the remaining four cases. In addition to proving approximation ratios and oracle complexities for several (challenging) settings that are the first or improvements over the state of the art, the technical novelties of our approach include:
(i) A new construction procedure of a shrunk constraint set that allows us to work with lower dimensional feasible sets when given a value oracle, resulting in the first results on general lower dimensional feasible sets given a value oracle.
(ii) The first Frank-Wolfe type algorithm for analyzing monotone functions over a general convex set for any type of oracle, where only feasible points can be queried.
(iii) Shedding light on a previously unexplained gap in approximation guarantees for monotone DR-submodular maximization. Specifically, by considering the notion of query sets and assuming that the oracles can only be queries within the constraint set, we divide the class of
monotone submodular maximization into monotone submodular maximization over convex sets containing the origin and monotone submodular maximization over general convex sets. Moreover, we conjecture that the  $1/2$  approximation coefficient, which has been considered sub-optimal in the literature, is optimal when oracle queries can only be made within the constraint set. (See Appendix B for more details.)
Furthermore, we also consider online stochastic DR-submodular optimization with bandit feedback, where an agent sequentially picks actions (from a convex feasible region), receives stochastic rewards (in expectation a DR-submodular function) but no additional information, and seeks to maximize the expected cumulative reward. Performance is measured against the best action in expectation (or a near-optimal baseline when the offline problem is NP-hard but can be approximated to within  $\alpha$  in polynomial time), the difference denoted as expected  $\alpha$ -regret. For such problems, when only bandit feedback is available (it is typically a strong assumption that semi-bandit or full-information feedback is available), the agent must be able to learn from stochastic value oracle queries over the feasible actions action. By designing offline algorithms that only query feasible points, we made it possible to convert those offline algorithms into online algorithms. In fact, because of how we designed the offline algorithms, we are able to access them in a black-box fashion for online problems when only bandit feedback is available. Note that previous works on DR-submodular maximization with bandit feedback in monotone settings (e.g. [31], [25] and [30]) explicitly assume that the convex set contains the origin.
For each of the offline setups, we extend the offline algorithm (the respective variants for stochastic value oracle) and oracle query guarantees to provide algorithms and  $\alpha$ -regret bounds in the bandit feedback scenario. Table 2 enumerates the problem settings and expected regret bounds with bandit and semi-bandit feedback. The key contributions of this work can be summarized as follows:
1. This paper proposes a unified approach for maximizing continuous DR-submodular functions in a range of settings with different oracle access types, feasible region properties, and function properties. A Frank-Wolfe based algorithm is introduced, which compared to SOTA methods for each of the sixteen settings, achieves the best-known approximation coefficients for each case while providing (i) the first guarantees in nine cases, (ii) reduced computational complexity by avoiding projections in three cases, and (iii) matching guarantees in remaining four cases.
2. In particular, this paper gives the first results on offline DR-submodular maximization (for both monotone and non-monotone functions) over general convex sets and even for downward-closed convex sets, when only a value oracle is available over the feasible set. Most prior works on offline DR-submodular maximization require access to a gradient oracle.
3. The results, summarized in Table 2, are presented with two feedback models—bandit feedback where only the (stochastic) reward value is available and semi-bandit feedback where a single stochastic sample of the gradient at the location is provided. This paper presents the first regret analysis with bandit feedback for stochastic DR-submodular maximization for both monotone and non-monotone functions. For semi-bandit feedback case, we provide the first result in one case, improve the state of the art result in two cases, and gives the result without computationally intensive projections in one case.
Table 2: Online stochastic DR-submodular optimization.
F	Set	Feedback	Reference	Coef. α	α-Regret
Monotone	0 ∈ K	∇F	[6]†,	1/e	O(T2/3)
This paper	1 - 1/e	O(T3/4)
F	This paper	1 - 1/e	O(T5/6)
general	∇F	[16]‡	1/2	O(T1/2)
This paper	1/2	O(T3/4)
F	This paper	1/2	O(T5/6)
Non-mono.	d.c.	∇F	This paper	1/e	O(T3/4)
F	This paper	1/e	O(T5/6)
general	∇F	This paper	1-h/4	O(T3/4)
F	This paper	1-h/4	O(T5/6)
This table compares the different results for the expected  $\alpha$ -regret for online stochastic DR-submodular maximization for the under bandit and semibandit feedback. Shaded rows indicate problem settings for which our work has the first guarantees or beats the SOTA. We have  $h := \min_{\mathbf{x} \in \mathcal{K}} \| x \|_{\infty}$ .  $\dagger$  the analysis in [6] has an error (see the supplementary material for details).  $\ddagger$  [16] uses gradient ascent, requiring potentially computationally expensive projections.
Related Work: The key related works are summarized in Tables 1 and 2. We briefly discuss some key works here; see the supplementary materials for more discussion. For online DR-submodular optimization with bandit feedback, there has been some prior works in the adversarial setup [31, 33, 25, 30] which are not included in Table 2 as we consider the stochastic setup. [31] considered monotone DR-submodular functions over downward-closed convex sets and achieved  $(1 - 1 / e)$ -regret of  $O(T^{8 / 9})$  in adversarial setting. This was later improved by [25] to  $O(T^{5 / 6})$ . [30] further improved the regret bound to  $O(T^{3 / 4})$ . However, it should be noted that they use a convex optimization subroutine at each iteration which could be even more computationally expensive than projection. [33] considered non-monotone DR-submodular functions over downward-closed convex sets and achieved  $1 / e$ -regret of  $O(T^{8 / 9})$  in adversarial setting.
2 Background and Notation
We introduce some basic notions, concepts and assumptions which will be used throughout the paper. For any vector  $\mathbf{x} \in \mathbb{R}^d$ ,  $[\mathbf{x}]_i$  is the  $i$ -th entry of  $\mathbf{x}$ . We consider the partial order on  $\mathbb{R}^d$  where  $\mathbf{x} \leq \mathbf{y}$  if and only if  $[\mathbf{x}]_i \leq [\mathbf{y}]_i$  for all  $1 \leq i \leq d$ . For two vectors  $\mathbf{x}, \mathbf{y} \in \mathbb{R}^d$ , the join of  $\mathbf{x}$  and  $\mathbf{y}$ , denoted by  $\mathbf{x} \vee \mathbf{y}$  and the meet of  $\mathbf{x}$  and  $\mathbf{y}$ , denoted by  $\mathbf{x} \wedge \mathbf{y}$ , are defined by
$$
\mathbf {x} \vee \mathbf {y} := \left(\max  \left\{\left[ \mathbf {x} \right] _ {i}, \left[ \mathbf {y} \right] _ {i} \right\}\right) _ {i = 1} ^ {d} \quad \text {a n d} \quad \mathbf {x} \wedge \mathbf {y} := \left(\min  \left\{\left[ \mathbf {x} \right] _ {i}, \left[ \mathbf {y} \right] _ {i} \right\}\right) _ {i = 1} ^ {d}, \tag {1}
$$
respectively. Clearly, we have  $\mathbf{x} \wedge \mathbf{y} \leq \mathbf{x} \leq \mathbf{x} \vee \mathbf{y}$ . We use  $\|\cdot\|$  to denote the Euclidean norm, and  $\|\cdot\|_{\infty}$  to denote the supremum norm. In the paper, we consider a bounded convex domain  $\mathcal{K}$  and w.l.o.g. assume that  $\mathcal{K} \subseteq [0,1]^d$ . We say that  $\mathcal{K}$  is down-closed (d.c.) if there is a point  $\mathbf{u} \in \mathcal{K}$  such that for all  $\mathbf{z} \in \mathcal{K}$ , we have  $\{\mathbf{x} \mid \mathbf{u} \leq \mathbf{x} \leq z\} \subseteq \mathcal{K}$ . The diameter  $D$  of the convex domain  $\mathcal{K}$  is defined as  $D := \sup_{\mathbf{x}, \mathbf{y} \in \mathcal{K}}\|\mathbf{x} - \mathbf{y}\|$ . We use  $\mathbb{B}_r(x)$  to denote the open ball of radius  $r$  centered at  $\mathbf{x}$ . More generally, for a subset  $X \subseteq \mathbb{R}^d$ , we define  $\mathbb{B}_r(X) := \bigcup_{x \in X} \mathbb{B}_r(x)$ . If  $A$  is an affine subspace of  $\mathbb{R}^d$ , then we define  $\mathbb{B}_r^A(X) := A \cap \mathbb{B}_r(X)$ . For a function  $F: \mathcal{D} \to \mathbb{R}$  and a set  $\mathcal{L}$ , we use  $F|_{\mathcal{L}}$  to denote the restriction of  $F$  to the set  $\mathcal{D} \cap \mathcal{L}$ . For a linear space  $\mathcal{L}_0 \subseteq \mathbb{R}^d$ , we use  $P_{\mathcal{L}_0}: \mathbb{R}^d \to \mathcal{L}_0$  to denote the projection onto  $\mathcal{L}_0$ . We will use  $\mathbb{R}_+^d$  to denote the set  $\{\mathbf{x} \in \mathbb{R}^d | \mathbf{x} \geq 0\}$ . For any set  $X \subseteq \mathbb{R}^d$ , the affine hull of  $X$ , denoted by  $\operatorname{aff}(X)$ , is defined to be the intersection of all affine subsets of  $\mathbb{R}^d$  that contain  $X$ . The relative interior of a set  $X$  is defined by
$$
\operatorname {r e l i n t} (X) := \{\mathbf {x} \in X \mid \exists \varepsilon > 0, \mathbb {B} _ {\varepsilon} ^ {\text {a f f} (X)} (\mathbf {x}) \subseteq X \}.
$$
It is well known that for any non-empty convex set  $\mathcal{K}$ , the set  $\mathrm{relint}(\mathcal{K})$  is always non-empty. We will always assume that the feasible set contains at least two points and therefore  $\dim(\operatorname{aff}(\mathcal{K})) \geq 1$ , otherwise the optimization problem is trivial and there is nothing to solve.
A set function  $f: \{0,1\}^d \to \mathbb{R}^+$  is called submodular if for all  $\mathbf{x}, \mathbf{y} \in \{0,1\}^d$  with  $\mathbf{x} \geq \mathbf{y}$ , we have
$$
f (\mathbf {x} \vee \mathbf {a}) - f (\mathbf {x}) \leq f (\mathbf {y} \vee \mathbf {a}) - f (\mathbf {y}), \quad \forall \mathbf {a} \in \{0, 1 \} ^ {d}. \tag {2}
$$
Submodular functions can be generalized over continuous domains. A function  $F:[0,1]^d \to \mathbb{R}^+$  is called  $DR$ -submodular if for all vectors  $\mathbf{x}, \mathbf{y} \in [0,1]^d$  with  $\mathbf{x} \leq \mathbf{y}$ , any basis vector  $\mathbf{e}_i = (0,\dots,0,1,0,\dots,0)$  and any constant  $c > 0$  such that  $\mathbf{x} + c\mathbf{e}_i \in [0,1]^d$  and  $\mathbf{y} + c\mathbf{e}_i \in [0,1]^d$ , it holds that
$$
F (\mathbf {x} + c \mathbf {e} _ {i}) - F (\mathbf {x}) \geq F (\mathbf {y} + c \mathbf {e} _ {i}) - F (\mathbf {y}). \tag {3}
$$
Note that if function  $F$  is differentiable then the diminishing-return (DR) property (3) is equivalent to  $\nabla F(\mathbf{x})\geq \nabla F(\mathbf{y})$  for  $\mathbf{x}\leq \mathbf{y}$  with  $\mathbf{x},\mathbf{y}\in [0,1]^d$ . A function  $F:\mathcal{D}\to \mathbb{R}^{+}$  is  $G$ -Lipschitz continuous if for all  $\mathbf{x},\mathbf{y}\in \mathcal{D}$ ,  $\| F(\mathbf{x}) - F(\mathbf{y})\| \leq G\| \mathbf{x} - \mathbf{y}\|$ . A differentiable function  $F:\mathcal{D}\rightarrow \mathbb{R}^{+}$  is  $L$ -smooth if for all  $\mathbf{x},\mathbf{y}\in \mathcal{D}$ ,  $\| \nabla F(\mathbf{x}) - \nabla F(\mathbf{y})\| \leq L\| \mathbf{x} - \mathbf{y}\|$ .
A (possibly randomized) offline algorithm is said to be an  $\alpha$ -approximation algorithm (for constant  $\alpha \in (0,1]$ ) with  $\epsilon \geq 0$  additive error for a class of maximization problems over non-negative functions if, for any problem instance  $\max_{\mathbf{z} \in \mathcal{K}} F(\mathbf{z})$ , the algorithm output  $\mathbf{x}$  that satisfies the following relation with the optimal solution  $\mathbf{z}^*$
$$
\alpha F \left(\mathbf {z} ^ {*}\right) - \mathbb {E} [ F (\mathbf {x}) ] \leq \epsilon , \tag {4}
$$
where the expectation is with respect to the (possible) randomness of the algorithm. Further, we assume an oracle that can query the value  $F(\mathbf{x})$  or the gradient  $\nabla F(\mathbf{x})$ . The number of calls to the oracle to achieve the error in (4) is called the evaluation complexity.
3 Offline Algorithms and Guarantees
In this section, we consider the problem of maximizing a DR-submodular function over a general convex set in sixteen different cases, enumerated in Table 1. After setting up the problem in Section 3.1, we then explain two key elements of our proposed algorithm when we only have access to a value oracle, (i) the Black Box Gradient Estimate (BBGE) procedure (Algorithm 1) to balance bias and variance in estimating gradients (Section 3.2) and (ii) the construction of a shrunken feasible region to avoid infeasible value oracle queries during the BBGE procedure (Section 3.3). Our main algorithm is proposed in Section 3.4 and analyzed in Section 3.5.
We consider a general non-oblivious constrained stochastic optimization problem
$$
\max  _ {\mathbf {z} \in \mathcal {K}} F (\mathbf {z}) := \max  _ {\mathbf {z} \in \mathcal {K}} \mathbb {E} _ {\mathbf {x} \sim p (\mathbf {x}; \mathbf {z})} [ \hat {F} (\mathbf {z}, \mathbf {x}) ], \tag {5}
$$
where  $F$  is a DR-submodular function, and  $\hat{F}:\mathcal{K}\times \mathfrak{X}\to \mathbb{R}$  is determined by  $\mathbf{z}$  and the random variable  $\mathbf{x}$  which is independently sampled according to  $\mathbf{x}\sim p(\mathbf{x};\mathbf{z})$ . We say the oracle has variance  $\sigma^2$  if  $\sup_{\mathbf{z}\in \mathcal{K}}\operatorname{var}_{\mathbf{x}\sim p(\mathbf{x};\mathbf{z})}[\hat{F} (\mathbf{z},\mathbf{x})] = \sigma^2$ . In particular, when  $\sigma = 0$ , then we say we have access to an exact (deterministic) value oracle. We will use  $\hat{F} (\mathbf{z})$  to denote the random variables  $\hat{F} (\mathbf{z},\mathbf{x})$  where  $\mathbf{x}$  is a random variable with distribution  $p(:,\mathbf{z})$ . Similarly, we say we have access to a stochastic gradient oracle if we can sample from function  $\hat{G}:\mathcal{K}\times \mathfrak{Y}\rightarrow \mathbb{R}$  such that  $\nabla F(\mathbf{z}) = \mathbb{E}_{\mathbf{y}\sim q(\mathbf{y};\mathbf{z})}[\hat{G} (\mathbf{z},\mathbf{y})]$ , and  $\hat{G}$  is determined by  $\mathbf{z}$  and the random variable  $\mathbf{y}$  which is sampled according to  $\mathbf{y}\sim q(\mathbf{y};\mathbf{z})$ . Note that oracles are only defined on the feasible set. We will use  $\hat{G} (\mathbf{z})$  to denote the random variables  $\hat{G} (\mathbf{z},\mathbf{y})$  where  $\mathbf{y}$  is a random variable with distribution  $q(:,\mathbf{z})$ .
Assumption 1. We assume that  $F:[0,1]^d\to \mathbb{R}$  is DR-submodular, first-order differentiable, nonnegative,  $G$ -Lipschitz for some  $G < \infty$ , and  $L$ -smooth for some  $L < \infty$ . We also assume the feasible region  $\mathcal{K}$  is a closed convex domain in  $[0,1]^d$  with at least two points. Moreover, we also assume that we either have access to a value oracle with variance  $\sigma_0^2\geq 0$  or a gradient oracle with variance  $\sigma_1^2\geq 0$ .
Remark 1. The proposed algorithm does not need to know the values of  $L$ ,  $G$ ,  $\sigma_0$  or  $\sigma_1$ . However, these constants appear in the final expressions of the number of oracle calls and the regret bounds.
3.2 Black Box Gradient Estimate
Without access to a gradient oracle (i.e., first-order information), we estimate gradient information using samples from a value oracle. We will use a variation of the "smoothing trick" technique [14, 17, 1, 27, 31, 8, 33], which involves averaging through spherical sampling around a given point.
Definition 1 (Smoothing Trick). For a function  $F: \mathcal{D} \to \mathbb{R}$  defined on  $\mathcal{D} \subseteq \mathbb{R}^d$ , its  $\delta$ -smoothed version  $\tilde{F}_{\delta}$  is given as
$$
\tilde {F} _ {\delta} (\mathbf {x}) := \mathbb {E} _ {\mathbf {z} \sim \mathbb {B} _ {\delta} ^ {\mathrm {a f f} (\mathcal {D})} (\mathbf {x})} [ F (\mathbf {z}) ] = \mathbb {E} _ {\mathbf {v} \sim \mathbb {B} _ {1} ^ {\mathrm {a f f} (\mathcal {D}) - \mathbf {x}} (0)} [ F (\mathbf {x} + \delta \mathbf {v}) ], \tag {6}
$$
where  $\mathbf{v}$  is chosen uniformly at random from the  $\dim (\mathrm{aff}(\mathcal{D}))$ -dimensional ball  $\mathbb{B}_1^{\mathrm{aff}(\mathcal{D}) - \mathbf{x}}(0)$ . Thus, the function value  $\tilde{F}_{\delta}(\mathbf{x})$  is obtained by "averaging"  $F$  over a sliced ball of radius  $\delta$  around  $\mathbf{x}$ .
When the value of  $\delta$  is clear from the context, we may drop the subscript and simply use  $\tilde{F}$  to denote the smoothed version of  $F$ . It can be easily seen that if  $F$  is DR-submodular,  $G$ -Lipschitz continuous, and  $L$ -smooth, then so is  $\tilde{F}$  and  $\| \tilde{F} (\mathbf{x}) - F(\mathbf{x})\| \leq \delta G$ , for any point in the domain of both functions. Moreover, if  $F$  is monotone, then so is  $\tilde{F}$  (Lemma 3). Therefore  $\tilde{F}_{\delta}$  is an approximation of the function  $F$ . A maximizer of  $\tilde{F}_{\delta}$  also maximizes  $F$  approximately.
Our definition of smoothing trick differs from the standard usage by accounting for the affine hull containing  $\mathcal{D}$ . This will be of particular importance when the feasible region is of (affine) dimension less than  $d$ , such as when there are equality constraints. When  $\operatorname{aff}(\mathcal{D}) = \mathbb{R}^d$ , our definition reduces to the standard definition of the smoothing trick. In this case, it is well-known that the gradient of the smoothed function  $\tilde{F}_{\delta}$  admits an unbiased one-point estimator [14, 17]. Using a two-point estimator instead of the one-point estimator results in smaller variance [1, 27]. In Algorithm 1, we adapt the two-point estimator to the general setting.
3.3 Construction of  $\kappa_{\delta}$