Project: Beam (App Definition and Development / Streaming & Messaging)
File: map.md

title: "Map" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> {{< localstorage language language-py >}} {{< button-pydoc path="apache_beam.transforms.core" class="Map" >}} Applies a simple 1-to-1 mapping function over each element in the collection. In the following examples, we create a pipeline with a `PCollection` of produce with their icon, name, and duration. Then, we apply `Map` in multiple ways to transform every element in the `PCollection`. `Map` accepts a function that returns a single element for every input element in the `PCollection`. We use the function `str.strip` which takes a single `str` element and outputs a `str`. It strips the input element's whitespaces, including newlines and tabs. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONMapSimple" show="mapsimple" >}} {{< /playground >}} We define a function `stripheaderand_newline` which strips any `'#'`, `' '`, and `'\n'` characters from each element. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONMapFunction" show="mapfunction" >}} {{< /playground >}} We can also use lambda functions to simplify Example 2. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONMapLambda" show="maplambda" >}} {{< /playground >}} You can pass functions with multiple arguments to `Map`. They are passed as additional positional arguments or keyword arguments to the function. In this example, `strip` takes `text` and `chars` as arguments. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONMapMultipleArguments" show="mapmultiple_arguments" >}} {{< /playground >}} If your `PCollection` consists of `(key, value)` pairs, you can use `MapTuple` to unpack them into different function arguments. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONMapTuple" show="maptuple" >}} {{< /playground >}} If the `PCollection` has a single value, such as the average from another computation, passing the `PCollection` as a singleton accesses that value. In this example, we pass a `PCollection` the value `'# \n'` as a singleton. We then use that value as the characters for the `str.strip` method. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONMapSideInputsSingleton" show="mapsideinputssingleton" >}} {{< /playground >}} If the `PCollection` has multiple values, pass the `PCollection` as an iterator. This accesses elements lazily as they are needed, so it is possible to iterate over large `PCollection`s that won't fit into memory. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONMapSideInputsIter" show="mapsideinputsiter" >}} {{< /playground >}} Note: You can pass the `PCollection` as a list with `beam.pvalue.AsList(pcollection)`, but this requires that all the elements fit into memory. If a `PCollection` is small enough to fit into memory, then that `PCollection` can be passed as a dictionary. Each element must be a `(key, value)` pair. Note that all the elements of the `PCollection` must fit into memory for this. If the `PCollection` won't fit into memory, use `beam.pvalue.AsIter(pcollection)` instead. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONMapSideInputsDict" show="mapsideinputsdict" >}} {{< /playground >}} behaves the same as `Map`, but for each input it may produce zero or more outputs. is useful if the function is just deciding whether to output an element or not. is the most general elementwise mapping operation, and includes other abilities such as multiple output collections and side-inputs. {{< button-pydoc path="apache_beam.transforms.core" class="Map" >}}