Spark is a general-purpose processing tool that works with Hadoop information. It can process information in Cassandra, HBase, HDFS, Hive, as well as any Hadoop InputFormat, and can operate in Hadoop networks via Spark's or YARN independent mode. It's built to handle batch jobs (much like MapReduce) as well as novel applications like broadcasting, machine learning and dynamic queries.