# A Virtual Learning Architecture Enhanced by Fog Computing and Big Data Streams
## 3 Big Data Streams and Fog Computing
### Big Data Streams
Apache S4 ([[http://incubator.apache.org/s4/](http://incubator.apache.org/s4/)]([http://incubator.apache.org/s4/](http://incubator.apache.org/s4/))), Apache Storm ([[http://storm.apache.org/](http://storm.apache.org/)]([http://storm.apache.org/](http://storm.apache.org/))) and Apache Samza ([[http://samza.apache.org/](http://samza.apache.org/)]([http://samza.apache.org/](http://samza.apache.org/))) are some Apache projects focusing on efficiently collecting, handling and managing big data streams. S4 is a Java-based platform, initially released by Yahoo!, capable of processing continuous streams of data in parallel. It is based on pluggable modules that can be combined in order to create complex processing systems. Storm is a distributed real-time computation system for quickly analyzing big streams of data. It is based on a master-slave approach and is composed by both a complex event processor and a distributed computation framework. Samza is a framework that processes stream messages as they arrive, one at a time. streams are divided into partitions that are in reality an ordered sequence of read-only messages. Samza relies on external facilities, for example Hadoop's YARN ([[http://hortonworks.com/apache/yarn/](http://hortonworks.com/apache/yarn/)]([http://hortonworks.com/apache/yarn/](http://hortonworks.com/apache/yarn/))) (Yet Another Resource Negotiator) and Apache Kafka ([[http://kafka.apache.org/](http://kafka.apache.org/)]([http://kafka.apache.org/](http://kafka.apache.org/))). The aforementioned technologies are simply stream processing engines that can be used as a basis for data mining and machine learning frameworks in order to extract useful knowledge from big data streams. Indeed, recently, the SAMOA ([[http://samoa.incubator.apache.org](http://samoa.incubator.apache.org)]([http://samoa.incubator.apache.org](http://samoa.incubator.apache.org))) (Scalable Advanced Massive Online Analysis) framework has been released in order to run distributed machine learning algorithms on stream processing engines, such as the aforementioned S4, Storm and Samza. Actually, the number of frameworks that allow handling both static big data and big data streams is increasing day by day. Recently, Apache Flink ([[http://flink.apache.org/](http://flink.apache.org/)]([http://flink.apache.org/](http://flink.apache.org/))), which provides data distribution, communication and fault tolerance for distributed computations over data streams, including also a machine learning library, has been introduced. Furthermore, Apache Spark ([[http://spark.apache.org/](http://spark.apache.org/)]([http://spark.apache.org/](http://spark.apache.org/))) has its own module for handling data streaming and a library of machine learning algorithms, namely MLib ([[http://spark.apache.org/mllib/](http://spark.apache.org/mllib/)]([http://spark.apache.org/mllib/](http://spark.apache.org/mllib/))). To the best of our knowledge, \(H_{2}O\) ([[http://www.h2o.ai](http://www.h2o.ai)]([http://www.h2o.ai](http://www.h2o.ai))) is the most recent open-source framework that provides a parallel processing engine, analytics and machine learning libraries, along with data pre-processing and evaluation tools. Finally, where stream management is concerned, also the way data are stored must be considered, and this leads to considering the so-called Data Stream Management Systems (DSMS), capable of performing continuous queries over streams of data, being infinite in size and thus unbounded in length.