Title: Universal Perfect Samplers for Incremental Streams
Domain: Computer Science

Question: How can we develop exact G-sampling algorithms for functions G in the specific class G (including p-th moments and logarithmic functions) when the vector x is presented as an incremental stream of positive updates, aiming to achieve exact sampling with limited memory and potentially supporting sampling a sequence of k indices with or without replacement?

Background: - The G-sampling problem involves a function G: R+ → R+. For a vector x ∈ R+n, the G-moment is defined as G(x) = ∑v∈[n]G(x(v)). The goal of G-sampling is to select an index v* ∈ [n] such that pr(v* = v) = G(x(v))/G(x). | | - Approximate G-samplers may introduce multiplicative and additive errors in the probability, and can have a non-trivial probability of failure. | | - The class of functions G includes Laplace exponents of non-negative, one-dimensional Lévy processes. It encompasses several well-studied classes such as pth moments G(z) = z^p for p ∈ [0, 1], logarithms G(z) = log(1 + z), and Cohen and Geri's soft concave sublinear functions used to approximate concave sublinear functions like cap statistics. | | - For G-sampling from incremental streams, Min-sketch and reservoir sampling can be used for F₀ and F₁ frequency moments.

Definitions: ### Definition 1 (Approximate/Perfect/Truly Perfect G-samplers) | Let \( G : \mathbb{R}_+ \to \mathbb{R}_+ \) be a function. An approximate G-sampler with parameters \( (\epsilon, \eta, \delta) \) is a sketch of \( \mathbf{x} \) that can produce an index \( v^* \in [n] \cup \{\perp\} \) such that \( \Pr(v^* = \perp) \leq \delta \) (where \( v^* = \perp \) indicates failure) and | \[ | \Pr(v^* = v \mid v^* \neq \perp) \in (1 \pm \epsilon)\frac{G(\mathbf{x}(v))}{G(\mathbf{x})} \pm \eta. | \] | If \( (\epsilon, \eta) = (0, 1/\text{poly}(n)) \), the sampler is said to be perfect. If \( (\epsilon, \eta) = (0, 0) \), it is truly perfect. | | ### Definition 2 (Non-negative Lévy processes) | A random process \( X = (X_t)_{t \geq 0} \) is a non-negative Lévy process if it satisfies: | 1. **Non-negativity**: \( X_t \in \mathbb{R}_+ \cup \{\infty\} \) for all \( t \in \mathbb{R}_+ \). | 2. **Stationary Increments**: \( X_{t+s} - X_t \sim X_s \) for all \( t, s \in \mathbb{R}_+ \). | 3. **Independent Increments**: For any \( 0 \leq t_1 < t_2 < \ldots < t_k \), \( X_{t_1}, X_{t_2} - X_{t_1}, \ldots, X_{t_k} - X_{t_{k-1}} \) are mutually independent. | 4. **Stochastic Continuity**: \( X_0 = 0 \) almost surely and \( \lim_{t \searrow 0} \Pr(X_t > \epsilon) = 0 \) for any \( \epsilon > 0 \). | | ### Definition 3 (Lévy-induced level function) | Let \( G \in \mathcal{G} \) and \( X = (X_t)_{t \geq 0} \) be the corresponding non-negative Lévy process. The induced level function \( \ell_G : (0, \infty) \times (0, 1) \to \mathbb{R}_+ \) is defined, for \( a \in (0, \infty) \) and \( b \in (0, 1) \), as | \[ | \ell_G(a, b) = \inf\{ t \mid \Pr(X_t \geq a) \geq b \}. | \]

Methodology: ## Leveraging Lévy Process Properties | | Since the function class G corresponds to non-negative Lévy processes, examining the properties of Lévy processes and related functions offers a novel perspective and theoretical foundation for constructing samplers. The Lévy-Khintchine representation theorem is especially useful as it links function G with non-negative Lévy processes, enabling the definition of the Lévy-induced level function ℓG, which is crucial for perfect sampling. | | ## Design of the General G-Sampler | | A general G-sampler has been developed, utilizing only 2 words of memory and capable of always returning an index v∗ according to the precise probability distribution of G(x(v))/G(x). | | ### Update Process | | When handling each update operation (v,∆), a new exponentially distributed random variable Y is generated. Based on the current update's increment ∆ and the level function ℓG of function G, a new value h is computed. If this h is less than the currently stored minimum h∗, the stored index and corresponding h value are updated. | | ## Design of ParetoSampler | | ParetoSampler employs O(logn) words of memory (with high probability) and can generate exact G samples for any given G∈G at query time. | | ### Maintenance of the Pareto Frontier | | During each update, tuples related to the update are generated, and the minimal Pareto frontier of these tuples is maintained. When sampling is required, the corresponding tuple is selected from the Pareto frontier based on the given G to determine the sampled index.

Answer: Exact G-sampling from positive-update streams | | Goal. Given a vector x ∈ Rⁿ₊ arriving as an incremental stream of positive updates (v, Δ), sample indices exactly with probability proportional to G(x(v)), for any G in the class | G(z) = c·1[z>0] + γ₀ z + ∫₀^∞ (1 − e^{−rz}) ν(dr) | (where c, γ₀ ≥ 0 and ν is a Lévy measure). This class includes z^p (p∈(0,1]), 1−e^{−τz}, and log(1+z). We also want limited memory and the ability to output k indices with or without replacement. | | Below is a clean recipe that achieves truly perfect sampling (zero error probability) with small space. | | ⸻ | | Key tool: Lévy-induced level functions | | For any G ∈ G, there exists a deterministic level function ℓ_G(a, b) such that: | • If Y ~ Exp(λ) and U ~ Uniform(0,1), then ℓ_G(Y, U) ~ Exp(G(λ)). | • ℓ_G is monotone increasing in both a and b. | | Interpretation: ℓ_G converts standard exponentials into exponentials whose rate is G(·). This is the engine behind exact sampling. | | ⸻ | | One-sample, O(1) state: Generic G-Sampler | | Maintain just a pair (v*, h*), initialized as (⊥, +∞). Use a pairwise-independent hash H : [n] → [0,1]. | | On update (v, Δ): | 1. Draw fresh Y ~ Exp(1). | 2. Compute h = ℓ_G(Y/Δ, H(v)). | 3. If h < h*, set (v*, h*) ← (v, h). | | Guarantee. At all times, P[v* = v] = G(x(v)) / G(∑_u x(u)). | Intuition: per index v, the minimum over its updates, | | h_v = min_i ℓ_G( (Y_i)/Δ_i , H(v) ) ~ Exp(G(x(v))) | | and the global min over v is an exponential race with total rate G(∑ x); the winner is sampled with exact G-weights. | | Space. Two words (plus the hash seed). | | ⸻ | | Many G’s, one pass, small space: ParetoSampler | | We can be universal over the whole class G by storing only the Pareto frontier of points | | (a_i, b_i, v_i) = (Y_i/Δ_i, H(v_i), v_i) | | ordered by (a, b). | | Update: insert (Y/Δ, H(v), v) and keep only the frontier. | Query(G): return the v attached to the frontier point that minimizes ℓ_G(a, b). | | Guarantee. For any G ∈ G, the returned v is an exact G-sample. | Space. Frontier size is O(log n) w.h.p., so space is O(log n) words after poly(n) updates. | | ⸻ | | k samples without replacement | | We need the k smallest keys among { h_v ~ Exp(G(x(v))) }. | | Streaming algorithm (heap of size k): | • For each update (v, Δ), compute h_candidate = ℓ_G(Y/Δ, H(v)) as before. | • Maintain, per index v, the current best key h_v = min of its candidates. | • Maintain a max-heap of the global k smallest distinct h_v. Replace the heap top if a new h_v beats it. | • At any time, output the heap’s k indices: this is the exact k-sample without replacement under G-weights (the order in the heap is the correct exponential-race order). | | Space. O(k) keys plus bookkeeping; in practice O(k + log n) words. | | (Why correct?) Independent exponentials with rates G(x(v)) order the indices exactly as sampling without replacement with those weights. | | ⸻ | | k samples with replacement | | Two standard exact options: | 1. Independent rounds. Repeat the one-sample algorithm k times using independent hash “salts” H₁, …, H_k (or disjoint RNG streams). Each round returns an exact G-weighted draw; duplicates can occur. | • Space: O(k) (one (v*, h*) per round) or O(k log n) if using the Pareto frontier per round. | 2. Poisson clocks (exponential races). For each v, generate an infinite sequence of i.i.d. keys { h_v^(j) ~ Exp(G(x(v))) } using independent salts; merge globally and take the first k arrivals with replacement. Streaming-wise, maintain the current k smallest among all generated keys. | • Space: O(k) keys; cost is extra randomness per additional arrival. | | Both methods give exact G-weighted sampling with replacement. | | ⸻ | | Instantiations for common G | | You get explicit ℓ_G and drop them straight into the update rules: | • p-th moments (G(z) = z^p, 0 < p ≤ 1). Use the subordinator for a stable law; this yields a closed-form ℓ_G. The special case p=1/2 gives the neat update: | | h = sqrt(2Y/Δ) · erf⁻¹(H(v)) | | (the F_{1/2} sampler). This produces exact p-weighted samples. | | • Poisson weight (G(z) = 1 − e^{−τz}): corresponds to a Poisson process; ℓ_G is a simple transform of Y/Δ and H(v). | • Logarithmic weight (G(z) = log(1+z)): corresponds to a Gamma subordinator; again ℓ_G is explicit via its CDF inversion. | | Because the class G matches Laplace exponents of non-negative Lévy processes, deriving ℓ_G is systematic: find the subordinator for G, define | ℓ_G(a, b) = inf{ t : P(X_t ≥ a) ≥ b }, or directly invert the map (Y, U) ↦ ℓ_G(Y, U) so that its output is Exp(G(·)). | | ⸻ | | Memory, updates, and guarantees | • Updates: Each stream update touches constant state: draw one exponential, one hash, one ℓ_G, and maybe a heap replace. | • Space: | • 1 sample → 2 words. | • Universal over G → O(log n) words w.h.p. (Pareto frontier). | • k without replacement → O(k) words; with replacement via independent rounds → O(k) as well. | • Correctness: All samplers are exact (no approximation, no failure prob): (ε, η, δ) = (0, 0, 0). | | ⸻ | | TL;DR (what to implement) | • Pick your G and its ℓ_G. | • For each (v, Δ): | • draw Y ~ Exp(1), compute h = ℓ_G(Y/Δ, H(v)), | • update (v*, h*) (one sample), the Pareto frontier (universal), or the top-k heap (k-sampling). | • Query at any time to return an exact G-weighted sample (or k of them).

Checklist: 1. **Correctly Define the Function Class G**  
   Ensure that the function class G is defined as  
   \[
   G(z) = c·1[z>0] + γ_0 z + \int_0^\infty (1 - e^{-rz}) \, \nu(dr),
   \]  
   where c, γ₀ ≥ 0 and ν is a non-negative Lévy measure with ∫ min(r,1)ν(dr) < ∞.  
   Verify its interpretation as the class of Laplace exponents of non-negative one-dimensional Lévy processes, and confirm that it includes examples such as \(G(z) = z^p\) (p ∈ (0,1]), \(G(z) = 1 - e^{−τz}\), and \(G(z) = \log(1+z)\).  

2. **Correctly Explain the Properties of Non-negative Lévy Processes**  
   Review the definition of non-negative Lévy processes (stationary and independent increments, stochastic continuity, non-negativity) and explain how the Lévy–Khintchine theorem establishes a one-to-one correspondence between any \(G ∈ G\) and a subordinator process with parameters (c, γ₀, ν).  

3. **Correctly Interpret the Lévy-induced Level Function ℓG**  
   Verify the definition  
   \[
   ℓ_G(a, b) = \inf\{t : P(X_t ≥ a) ≥ b\},
   \]  
   where \(X_t\) is the Lévy process associated with G.  
   Confirm that ℓG is monotone increasing in both arguments and that applying ℓG to exponential/Uniform random variables transforms Exp(λ) into Exp(G(λ)), which is the key to perfect sampling.  

4. **Correctly Describe the Function and Storage Efficiency of the General G-Sampler**  
   State that the generic G-Sampler (Algorithm 1) maintains only a pair (v*, h*) in memory, i.e. 2 words, and guarantees that at all times \(P[v* = v] = G(x(v))/G(x)\).  

5. **Correctly Explain the Update Process of the General G-Sampler**  
   Detail that for each update (v, Δ), the sampler:  
   - draws Y ~ Exp(1),  
   - computes h = ℓG(Y/Δ, H(v)),  
   - updates (v*, h*) if h is smaller.  
   Verify that per-index minima h_v follow Exp(G(x(v))) and the global winner follows the exponential race with rate G(x).  

6. **Correctly State the Storage Efficiency and Sampling Performance of ParetoSampler**  
   Confirm that ParetoSampler keeps only the Pareto frontier of tuples (Y/Δ, H(v), v). Its expected size is < ln n+1 and O(log n) with high probability, so total space is O(log n) words, while still allowing exact sampling for any G at query time.  

7. **Correctly Interpret the Pareto Frontier Maintenance Mechanism of ParetoSampler**  
   Ensure that each update inserts (Y/Δ, H(v), v) and prunes dominated points. At query time, given any G, the chosen index is the one attached to the frontier point with minimal ℓG(a, b).  

8. **Correctly Present and Prove Lemma 1 (Level Function Lemma)**  
   State: For any G ∈ G, there exists ℓG such that  
   - **2D-monotonicity**: if a ≤ a′ and b ≤ b′ then ℓG(a, b) ≤ ℓG(a′, b′),  
   - **G-transformation**: if Y ~ Exp(λ) and U ~ Uniform(0,1), then ℓG(Y, U) ~ Exp(G(λ)).  
   Proof relies on the Lévy–Khintchine representation, showing that the distribution of ℓG(Y, U) matches Exp(G(λ)).  

9. **Correctly Present and Prove Theorems 1 and 2 (Correctness of Samplers)**  
   - **Theorem 1 (G-Sampler)**: The generic sampler always stores a pair (v*, h*) such that h* ~ Exp(G(x)) and P[v* = v] = G(x(v))/G(x), hence it is a truly perfect sampler.  
   - **Theorem 2 (ParetoSampler)**: ParetoSampler uses O(log n) space (w.h.p.) and for any G ∈ G produces exact samples with the same distribution as G-Sampler.  
   Proofs use Lemma 1 and exponential race arguments.  

10. **Correctly Derive the Expressions for ℓG for Specific Functions G**  
    Show explicit forms:  
    - \(G(z) = 1[z>0]\): ℓG(a, b) = −log(1−b) (F₀ sampler).  
    - \(G(z) = z\): ℓG(a, b) = a (F₁ sampler).  
    - \(G(z) = √z\): ℓG(a, b) = √(2a) · erf⁻¹(b) (F1/2 sampler).  
    - \(G(z) = 1−e^{−τz}\): ℓG found by solving Poisson tail equations.  
    - \(G(z) = log(1+z)\): ℓG found as solution of Gamma CDF equations.  

11. **Correctly Present the Method for Sampling Without Replacement and Provide Theoretical Guarantees**  
    State that the (G, k)-Sampler-WOR algorithm maintains the k smallest keys {(v, h)} under ℓG.  
    **Theorem 4**: Its output distribution satisfies  
    \[
    P((v_1,…,v_k)) = \prod_{i=1}^k \frac{G(x(v_i))}{G(x) - \sum_{j<i} G(x(v_j))},
    \]  
    exactly matching sampling k indices without replacement. Space usage is 2k words for (G,k)-Sampler-WOR or O(k log n) for k-ParetoSampler-WOR.