Systems and Architecture Support for Large-Scale Video Search

To index and search all of the videos in the Internet requires massive resources. Using commodity parts and existing libraries, a conservative estimate for a system requires 32 petabytes to store feature data, 480,000 CPU cores to build and update the index, and 1000 servers to service 1 million concurrent clients at a latency on the order of seconds. Such requirements in both power consumption and performance rival those of the top supercomputers [1]. While it may be feasible to add commodity CPUs to increase raw processing power, the exponential rates of multimedia content creation and consumption exacerbate the scalability of network infrastructure and memory subsystems. In 2013 alone, approximately 127 billion images and 620 million videos were uploaded to Facebook and YouTube respectively [2, 3]. In the same year, traffic volumes from Netflix and YouTube accounted for more than 50% of global Internet traffic [4]. This growth in data has created a landscape where multimedia search engines will be essential to organize and reason about the available data. To date, advances in feature extraction and visual search algorithms have enabled image-based search engines. However, increased I/O and compute requirements have stymied the progress towards video-based search engines. To enable video search at scale, rearchitecting the roles of the memory subsystem and network layer is imperative to address the challenges we encounter below. I Feature Storage and Retrieval. The core search algorithm is highly memory bound. Accessing feature data quickly is critical to satisfying low-latency demands in user-facing applications. We propose to: (1) exploit locality by intelligently sharding similar feature vectors and (2) minimize data movement across the networking layer and within the memory subsystem. II Indexing. All feature vectors are indexed into an efficient search data structure which enables sublinear search time scaling with respect to data size. The one time cost of indexing is amortized by the number of searches performed over it. Indexing is three orders of magnitude slower than search time, and while search time grows linearly with the dimensionality of the data, the indexing time grows linearly with both dimensionality and cardinality. The index must also adapt and be rebuilt when new video content is uploaded to or deleted from the database. We propose to: (1) design a scalable, distributed indexing data structure for video search, (2) develop a methodology and implementation for incremental reindexing, and (3) integrate application awareness into the system architecture to minimize data movement. To address the above challenges, we will combine application optimizations with hardware/software codesign techniques. Within a single machine or intra-node level (Section 2), we introduce architectural support for the application using in-memory computation and evaluate the potential of a lightweight accelerator design for the core search algorithm. At the cluster or inter-node level (Section 3), we propose to also codesign hardware and software subsystems to enable scalable, distributed data indexing with extensions for incremental indexing. We also propose integrating in-network application awareness to support intelligent routing of data. Many applications in Computer Vision, Robotics, and Machine Learning such as large-scale 3D reconstruction [5], point cloud perception [6], and classifier construction [7] rely on the same algorithms used in video search. Improvements in the scalability and performance of video search ultimately improve the performance of such data-intensive applications.