Paper: High-Dimensional Geometric Streaming for Nearly Low Rank Data (ICLR-2024)

Reviewer comment (Presentation):
The paper is quite technical and is not easy to understand especially for general audience. In addition, too many results are squeezed into the 9 pages. In my point of view, the author could focus on the main result Theorem 1.1, and this itself should already fit the volume of an ICLR paper (considering the 9 pages of the main text).

Author response:
- **Reorganizing the paper**: We will think about better ways to reorganize the paper to make it more understandable to the general audience. We think the applications to width estimation, convex hull approximation are important and should be present in the main paper and we will look for ways to make the presentation a bit less dense.

Reviewer comment (Novelty):
I don't see a related work section. Since your main technique is coreset, it might make sense to mention works related to coreset.

Author response:
- **Related work**: We will add a discussion about the existing coreset literature in the streaming/non-streaming settings for subspace approximation and related problems. We note that ours is the first strong coreset construction, with only poly(k, log n) points, for $\ell_{\infty}$ subspace approximation in the streaming setting and as we show the strong coreset property is important to obtain our applications.

Reviewer comment (Theory):
Does JL work here? In particular, can one reduce d to O(log n)? Or maybe subspace JL that reduce to O(k) dimension? I didn't find this discussed/mentioned. This is important as I guess otherwise your approach may not improve WS22? It may be useful to have a brief discussion of this in the paper.

Author response:
- **Projecting with JL**: It is not clear how to obtain strong coreset properties by projecting to lower dimensions using JL transforms in the streaming setting. Usually, projections with JL are used to obtain approximate sensitivities and then a strong coreset is constructed. But this cannot be implemented in the streaming setting with one pass. Kerber and Raghavendra (CCCG 2015) show that when we project to poly(k, log n) dimensions and construct a coreset, we can preserve the costs and extract a solution but we cannot approximate the cost of every subspace and obtain our application such as width estimation, convex hull etc.

Reviewer comment (Theory):
It seems Theorem 3.3 and Theorem 3.6 are in different models of streaming algorithms? Also, does the streaming algorithm in Theorem 3.3 work if deletions are allowed? Please clarify in the paper.

Author response:
- **Streaming Model**: All the results in our paper are in the row-arrival model where we see one point after another. Our results do not hold when points can later be deleted.

Reviewer comment (Theory):
In both Theorem 3.3 and Theorem 3.6 it is mentioned that the coordiantes are integers. Is this necessary even for the offline algorithm, or it is only a matter of storage model? Please clarify in the paper.

Author response:
- **Integer Points**: It is not necessary for the points to have integer coordinates. We use bounded integer assumption to only obtain results that are independent of the condition number of the data. However, the algorithms will work even for arbitrary values and the size of the coreset constructed will depend on a specific condition number of the dataset.

Reviewer comment (Theory):
For width approximation, there are no randomized algorithm can achieve better than $d^{1/3}$-approximation while using $e^{d^{1/3}}$ space, this was shown in Agarwal and Sharathkumar, (SODA 2010, Algorithmica 2015). I think it is important to place your result in the context of this lower bound.

Author response:
The width estimation problem, as studied in Agarwal and Sharathkumar (SODA 2014), corresponds to the case of k = d-1 for the $\ell_{\infty}$ subspace approximation problem. The hardness result of Agarwal and Sharathkumar is, as you stated, for approximating to a better factor than d^{1/3} in small space. Our streaming coreset construction gives only multiplicative approximation up to a factor of d^{3/2} in a bounded integer points case and a d^{1/2} factor for well-conditioned instances in this setting (k=d-1) and hence makes a poly(d) space streaming algorithm possible.

Reviewer comment (Novelty):
Overall, I had difficulty in understanding the impact and importance of this work and this, I believe, is mainly due to lack of a good comparison with existing work.

Author response:
We will recap our main contributions here:
- A **deterministic** coreset construction algorithm for $\ell_{\infty}$ subspace approximation with **any** rank parameter k. We are not aware of any previous works with streaming algorithms that work for any given rank parameter k.
- A black-box **randomized reduction** from an $\ell_p$ subspace approximation coreset to an $\ell_{\infty}$ subspace approximation coreset for any rank k using scaled exponential random variables. This reduction is also new and may have additional applications since any improvement in an $\ell_{\infty}$ coreset construction would **directly improve** the $\ell_p$ coreset construction as well.
- Deterministic Streaming Algorithms for other problems such as convex hull estimation, volume estimation, and width estimation in any direction, for almost low rank data with better guarantees than previous work of Woodruff and Yasuda (FOCS 2022)

Our main motivation was to obtain streaming algorithms that work for values of k that are not just small constants as well. This particular problem seems to have not been studied in the streaming setting. We will add references to the line of work studying streaming algorithms for k=0,1 and add a discussion to place our results in context.