We present an I O-efficient algorithm for computing similarity joins based on locality-sensitive hashing (LSH). In contrast to the filtering methods commonly suggested our method has provable sub-quadratic dependency on the data size. Further, in contrast to straightforward implementations of known LSH-based algorithms on external memory, our approach is able to take significant advantage of the available internal memory: Whereas the time complexity of classical algorithms includes a factor of @math , where @math is a parameter of the LSH used, the I O complexity of our algorithm merely includes a factor @math , where @math is the data size and @math is the size of internal memory. Our algorithm is randomized and outputs the correct result with high probability. It is a simple, recursive, cache-oblivious procedure, and we believe that it will be useful also in other computational settings such as parallel computation.
We study algorithms for similarity join in the , which has been widely adopted in the literature (see, e.g., the survey by Vitter @cite_22 ). The external memory model consists of an internal memory of @math words and an external memory of unbounded size. The processor can only access data stored in the internal memory and move data between the two memories in blocks of size @math . For simplicity we will here measure block and internal memory size in units of points from @math , such that they can contain @math points and @math points, respectively.
@cite_22
2072336782
A straight-line grid embedding of a planar graph is a drawing of the graph on a plane where the vertices are located at grid points and the edges are represented by nonintersecting segments of straight, lines joining their incident vertices. Given an n-vertex embedded planar graph with @math , a straight-line embedding on a grid of size @math can be computed deterministically in @math time with @math processors. If randomization is used, the complexity is improved to @math expected time with the same optimal linear work. These algorithms run on a parallel random access machine that allows concurrent reads and concurrent writes of the shared memory and permits an arbitrary processor to succeed in case of a write conflict. In this paper, we study the communication complexity for the problem of computing a conjunctive query on a large database in a parallel setting with @math servers. In contrast to previous work, where upper and lower bounds on the communication were specified for particular structures of data (either data without skew, or data with specific types of skew), in this work we focus on worst-case analysis of the communication cost. The goal is to find worst-case optimal parallel algorithms, similar to the work of [18] for sequential algorithms. We first show that for a single round we can obtain an optimal worst-case algorithm. The optimal load for a conjunctive query @math when all relations have size equal to @math is @math , where @math is a new query-related quantity called the edge quasi-packing number, which is different from both the edge packing number and edge cover number of the query hypergraph. For multiple rounds, we present algorithms that are optimal for several classes of queries. Finally, we show a surprising connection to the external memory model, which allows us to translate parallel algorithms to external memory algorithms. This technique allows us to recover (within a polylogarithmic factor) several recent results on the I O complexity for computing join queries, and also obtain optimal algorithms for other classes of queries. We consider streaming, one-pass principal component analysis (PCA), in the high-dimensional regime, with limited memory. Here, @math -dimensional samples are presented sequentially, and the goal is to produce the @math -dimensional subspace that best approximates these points. Standard algorithms require @math memory; meanwhile no algorithm can do better than @math memory, since this is what the output itself requires. Memory (or storage) complexity is most meaningful when understood in the context of computational and sample complexity. Sample complexity for high-dimensional PCA is typically studied in the setting of the spiked covariance model , where @math -dimensional points are generated from a population covariance equal to the identity (white noise) plus a low-dimensional perturbation (the spike) which is the signal to be recovered. It is now well-understood that the spike can be recovered when the number of samples, @math , scales proportionally with the dimension, @math . Yet, all algorithms that provably achieve this, have memory complexity @math . Meanwhile, algorithms with memory-complexity @math do not have provable bounds on sample complexity comparable to @math . We present an algorithm that achieves both: it uses @math memory (meaning storage of any kind) and is able to compute the @math -dimensional spike with @math sample-complexity -- the first algorithm of its kind. While our theoretical analysis focuses on the spiked covariance model, our simulations show that our algorithm is successful on much more general models for the data. This paper presents post-silicon results for the Ascend secure processor, taped out in a 32 nm SOI process. Ascend prevents information leakage over a processor's digital I O pins—in particular, the processor's requests to external memory—and certifies the program's execution by verifying the integrity of the external memory. In secure processor design, encrypting main memory is not sufficient for security because where and when memory is accessed reveals secret information. To this end, Ascend is equipped with a hardware Oblivious RAM (ORAM) controller, which obfuscates the address bus by reshuffling memory as it is accessed. To our knowledge, Ascend is the first prototyping of ORAM in custom silicon. Ascend has also been carefully engineered to ensure its timing behaviors are independent of user private data. In 32 nm silicon, all security components combined (the ORAM controller, which includes 12 AES rounds and one SHA-3 hash unit) impose a moderate area overhead of 0.51 mm @math 2 . Post tape-out, the security components of the Ascend chip have been successfully tested at 857 MHz and 1.1 V, at which point they consume 299 mW of power.