num_pages: 300
description: Data is bigger arrives faster and comes in a variety of formats--and it all needs to be processed at scale for analytics or machine learning. But how can you process such varied workloads efficiently? Enter Apache Spark.Updated to include Spark 3.0 this second edition shows data engineers and data scientists why structure and unification in Spark matters. Specifically this book explains how to perform simple and complex data analytics and employ machine learning algorithms. Through step-by-step walk-throughs code snippets and notebooks you'll be able to:Learn Python SQL Scala or Java high-level Structured APIsUnderstand Spark operations and SQL EngineInspect tune and debug Spark operations with Spark configurations and Spark UIConnect to data sources: JSON Parquet CSV Avro ORC Hive S3 or KafkaPerform analytics on batch and streaming data using Structured StreamingBuild reliable data pipelines with open source Delta Lake and SparkDevelop machine learning pipelines with MLlib and productionize models using MLflow
language: English
title: Learning Spark: Lightning-Fast Data Analytics
publisher: O'Reilly Media