SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Then, we update the index sets  $S_{i,\mathrm{fire}}$  in time  $O(m^{4/5}n)$ . Therefore, each training step takes  $O(m^{4/5}n^2d)$  time, which completes the proof.
F Lower Bound for Dynamic Detection of Firing Neurons
The goal of this section is to prove the lower bound for Dynamic Detection of Firing Neurons.
We start by introducing the strong exponential time hypothesis, SETH in abbreviation.
Definition F.1 (Strong exponential time hypothesis, SETH, [IP01, CIP09]). For every  $\epsilon > 0$ , there exists a  $k = k(\epsilon) \in \mathbb{N}$  such that no algorithm can solve  $k$ -SAT (i.e., satisfiability on a CNF of width  $k$ ) in  $O(2^{(1 - \epsilon)n})$  time where  $n$  is the number of variables.
We present another relative concept called orthogonal vector conjecture, OVC in abbreviation.
Definition F.2 (Orthogonal vector conjecture, OVC, [Wil05, AWW14, BI15, ABW15]). For every  $\epsilon > 0$ , there exists a  $c \geq 1$  such that the orthogonal vector problem of size  $n$  in  $d$ -dimension requires  $n^{2 - \epsilon}$ -time when  $d = c \log n$ .
We refer to a theorem about maximum bichromatic inner product lower bound in [Che20].
Theorem F.3 (Maximum bichromatic inner product lower bound, [Che20]). Assuming SETH (Definition F.1) or OVC (Definition F.2), there is a constant  $c$  such that any exact algorithm for  $\mathbb{Z}$ -Max- $IP_{n,d}$  in dimension  $d = c^{\log^{*}n}$  requires  $n^{2 - o(1)}$  time, with vectors of  $O(\log n)$ -bit entries.
Putting things together, we state the main result for the lower bound for Dynamic Detection of Firing Neurons.
Theorem F.4 (Lower Bound for Dynamic Detection of Firing Neurons, Formal version of Theorem 1.4). Let  $d = 2^{O(\log^* n)}$ . Unless OVC or SETH fails, for any constants  $c \in (0,1)$ , no data structure can solve DDFN with less than  $m^{1 - c}n^{c - o(1)}$ -time per update and  $m^{1 - c}n^{1 + c - o(1)}$ -time per query.
Proof. Without loss of generality, we assume that  $m > n$ . Let  $d = c^{\log^{*}m}$ , where  $c$  is defined in Theorem F.3.
Suppose there exists a data structure that for  $(m,n,d + 1)$ -sized instance, can perform updates in  $m^{1 - c}n^{c - \epsilon}$ -time and answer queries in  $m^{1 - c}n^{1 + c - \epsilon}$ -time, for some  $c\in (0,1)$  and  $\epsilon \in (0,c)$ .
Let  $X = \{x_{1},\ldots ,x_{m}\} \subset \mathbb{Z}^{d}$ ,  $Y = \{y_{1},\dots ,y_{m}\} \subset \mathbb{Z}^{d}$  be a hard instance of  $\mathbb{Z}$ -Max-IP  $m,d$  problem constructed in Theorem F.3. For each vector  $x_{i}$  (or  $y_{j}$ ), we construct a new vector  $\widetilde{x}_i$  (or  $\widetilde{y}_j$ ) in  $(d + 1)$ -dimension such that  $(\widetilde{x}_i)_{d + 1} = -1$  and  $(\widetilde{y}_j)_{d + 1} = w$ , where  $w$  is a parameter to be chosen later.
Then, we construct  $k = \lceil m / n \rceil$  instances of the DDFN problem in Definition 1.3 as follows:
$$
\widetilde {X} ^ {(i)} := \{\widetilde {x} _ {1}, \ldots , \widetilde {x} _ {n} \}, \widetilde {Y} ^ {(i)} := \{\widetilde {y} _ {1}, \ldots , \widetilde {y} _ {m} \},
$$
and  $b = 0$
We show that the data structures for these instances  $\{(\widetilde{X}^{(i)},\widetilde{Y}^{(i)},b)\}_{i\in [k]}$  can be used to solve  $\mathbb{Z}$ -Max-IP $_{n,d}(X,Y)$ .
We perform a binary search for the value of  $\mathbb{Z}$ -Max- $\mathsf{IP}_{n,d}(X,Y)$ . Note that at most  $O(\log n)$  iterations suffice to find the exact answer.
Suppose the current value in the binary search is  $t \in \mathbb{Z}$ . Consider the  $i$ -th instance  $(\widetilde{X}^{(i)},\widetilde{Y}^{(i)},b)$  for any  $i \in [k]$ . We first call UPDATE() to set  $(\widetilde{y}_j)_{d + 1} = t$  for each  $j \in [m]$ . By the data structure's guarantee, this step takes  $O(m \cdot m^{1 - c}n^{c - \epsilon}) = O(m^{2 - c}n^{c - \epsilon})$  time. Then, we call QUERY(). Notice that
$$
\langle \widetilde {x} _ {i}, \widetilde {y} _ {j} \rangle = \langle x _ {i}, y _ {j} \rangle - t \geq 0 \iff \langle x _ {i}, y _ {j} \rangle \geq t.
$$
Hence,  $\mathrm{QUERY}(i,j)$  will return all pairs of  $(i,j)$  such that  $\langle x_i,y_j\rangle \geq t$ . This step runs in  $O(m^{1 - c}n^{1 + c - \epsilon})$ -time. We repeat this process for all  $k$  instances. And based on whether the outputs of all the  $\mathrm{QUERY}(i)$  are empty or not, we know the direction of the binary search for the next iteration.
Hence, the total running time of each iteration is
$$
\begin{array}{l} O (k \cdot \left(m ^ {2 - c} n ^ {c - \epsilon} + m ^ {1 - c} n ^ {1 + c - \epsilon}\right)) \\ = O \left(m ^ {3 - c} n ^ {c - \epsilon - 1} + m ^ {2 - c} n ^ {c - \epsilon}\right) \\ \leq O (m ^ {2 - \epsilon}), \\ \end{array}
$$
which follows from the assumption of  $m \geq n$ . Thus, we can solve  $\mathbb{Z}$ -Max- $\mathsf{IP}_{n,d}(X,Y)$  in  $\widetilde{O}(m^{2 - \epsilon}) < m^{2 - o(1)}$ -time, which contradicts to the lower bound for  $\mathbb{Z}$ -Max- $\mathsf{IP}_{n,d}$  in Theorem F.3.
Therefore, no such data structure can exist.