From Data to Knowledge - 208 - Marc Berkowitz
Channel: ckleinastro
Date: 2012-05-09

Marc Berkowitz: "A Dynamically Scalable Cloud Platform for the Real-Time Detection of Seismic Events". A video from the UC Berkeley Conference: From Data to Knowledge: Machine-Learning with Real-time and Streaming Applications (May 7-11, 2012).

Abstract
Marc Berkowitz (SQLstream Inc.)
UCSD seismologists have deployed an application based on SQLstream that detects significant events in data collected from a large grid of seismic sensors. A large-scale data infrastructure (the OOI/CI) provides raw signal data over an AMQP message bus. SQLstream monitors this data in real-time, applying heuristic algorithms that look for patterns indicating earthquakes. The detected events are streamed in real-time back onto the AMQP message bus for visualization and further processing. The detection algorithms are coded in streaming SQL, with Java plugins for added time-series operations. The live system executes across multiple servers in a cloud environment. The number of servers is adjusted automatically, based on the current demand, by a control program in Python. SQLstream is a scalable, distributed platform for real-time monitoring and analysis of event streams. Applications are built using continuous SQL queries that process high speed, high volume data streams over moving time windows. Unlike traditional data management platforms that must store data before processing, SQLstream processes streams of arriving event data on the fly. Thus alerts and other analytics are generated continuously, without having to store the data.