Project: Beam (App Definition and Development / Streaming & Messaging)
File: python.md

title: "Beam Quickstart for Python" aliases: /get-started/quickstart/ /use/quickstart/ /getting-started/ <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> This quickstart shows you how to run an written with the , using the . The Direct Runner executes pipelines locally on your machine. If you're interested in contributing to the Apache Beam Python codebase, see the . On this page: {{< toc >}} Apache Beam aims to work on released that have not yet reached end of life, but it may take a few releases until Apache Beam fully supports the most recently released Python minor version. The minimum required Python version is listed in the Meta section of the project page under Requires. The list of all supported Python versions is listed in the Classifiers section at the bottom of the page, under Programming Language. Check your Python version by running: {{< highlight >}} python3 --version {{< /highlight >}} If you don't have a Python interpreter, you can download and install it from the page. If you need to install a different version of Python in addition to the version that you already have, you can find some recommendations in our . Clone or download the GitHub repository and change into the `beam-starter-python` directory. {{< highlight >}} git clone https://github.com/apache/beam-starter-python.git cd beam-starter-python {{< /highlight >}} A virtual environment is a directory tree containing its own Python distribution. We recommend using a virtual environment so that all dependencies of your project are installed in an isolated and self-contained environment. To set up a virtual environment, run the following commands: {{< highlight >}} python3 -m venv env source env/bin/activate {{< /highlight >}} If these commands do not work on your platform, see the documentation. Run the following command to install the project's dependencies from the `requirements.txt` file: {{< highlight >}} pip install -e . {{< /highlight >}} Run the following command: {{< highlight >}} python

--input-text="Greetings" {{< /highlight >}} The output is similar to the following: {{< highlight >}} Hello World! Greetings {{< /highlight >}} The lines might appear in a different order. Run the following command to deactivate the virtual environment: {{< highlight >}} deactivate {{< /highlight >}} The main code file for this quickstart is app.py (). The code performs the following steps: Create a Beam pipeline. Create an initial `PCollection`. Apply a transform to the `PCollection`. Run the pipeline, using the Direct Runner. The code first creates a `Pipeline` object. The `Pipeline` object builds up the graph of transformations to be executed. ```python with beam.Pipeline(options=beam_options) as pipeline: ``` The `beam_option` variable shown here is a `PipelineOptions` object, which is used to set options for the pipeline. For more information, see . The `PCollection` abstraction represents a potentially distributed, multi-element data set. A Beam pipeline needs a source of data to populate an initial `PCollection`. The source can be bounded (with a known, fixed size) or unbounded (with unlimited size). This example uses the method to create a `PCollection` from an in-memory array of strings. The resulting `PCollection` contains the strings "Hello", "World!", and a user-provided input string. ```python pipeline | "Create elements" >> beam.Create(["Hello", "World!", input_text]) ``` Note: The pipe operator `|` is used to transforms. Transforms can change, filter, group, analyze, or otherwise process the elements in a `PCollection`. This example uses the transform, which maps the elements of a collection into a new collection: ```python | "Print elements" >> beam.Map(print) ``` To run the pipeline, you can call the method: ```python pipeline.run.waituntilfinish() ``` However, by enclosing the `Pipeline` object inside a `with` statement, the `run` method is automatically invoked. ```python with beam.Pipeline(options=beam_options) as pipeline: ``` A Beam runs a Beam pipeline on a specific platform. If you don't specify a runner, the Direct Runner is the default. The Direct Runner runs the pipeline locally on your machine. It is meant for testing and development, rather than being optimized for efficiency. For more information, see . For production workloads, you typically use a distributed runner that runs the pipeline on a big data processing system such as Apache Flink, Apache Spark, or Google Cloud Dataflow. These systems support massively parallel processing. Learn more about the and look through the . Take a self-paced tour through our . Dive in to some of our favorite . Join the Beam mailing list. Please don't hesitate to if you encounter any issues!