Module Expr Str Import alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias
N/A N/A Google Cloud Dataflow SDK for Python.

Google Cloud Dataflow <http://cloud.google.com/dataflow/>
provides a simple, powerful programming model for building both batch
and streaming parallel data processing pipelines.

The Dataflow SDK for Python provides access to Dataflow capabilities
from the Python programming language.

Status
------
The SDK is still early in its development, and significant changes
should be expected before the first stable version.

Overview
--------
The key concepts in this programming model are

* PCollection:  represents a collection of data, which could be
  bounded or unbounded in size.
* PTransform:  represents a computation that transforms input
  PCollections into output PCollections.
* Pipeline:  manages a directed acyclic graph of PTransforms and
  PCollections that is ready for execution.
* Runner:  specifies where and how the Pipeline should execute.
* Reading and Writing Data:  your pipeline can read from an external
  source and write to an external data sink.

Typical usage
-------------
At the top of your source file::

    import google.cloud.dataflow as df

After this import statement

* transform classes are available as df.FlatMap, df.GroupByKey, etc.
* Pipeline class is available as df.Pipeline
* text source/sink classes are available as df.io.TextFileSource,
  df.io.TextFileSink

Examples
--------
The examples subdirectory has some examples.

 N/A google.cloud.dataflow.internal.pickler google.cloud.dataflow coders google.cloud.dataflow io google.cloud.dataflow typehints google.cloud.dataflow.pipeline Pipeline google.cloud.dataflow.transforms *