Project: Beam (App Definition and Development / Streaming & Messaging)
File: regex.md

title: "Regex" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> {{< localstorage language language-py >}} {{< button-pydoc path="apache_beam.transforms.util" class="Regex" >}} Filters input string elements based on a regex. May also transform them based on the matching groups. In the following examples, we create a pipeline with a `PCollection` of text strings. Then, we use the `Regex` transform to search, replace, and split through the text elements using . You can use tools to help you create and test your regular expressions, such as . Make sure to specify the Python flavor at the left side bar. Lets look at the for example. It matches anything that is not a whitespace `\s` (`[ \t\n\r\f\v]`) or comma `,` until a comma is found and stores that in the named group `icon`, this can match even `utf-8` strings. Then it matches any number of whitespaces, followed by at least one word character `\w` (`[a-zA-Z0-9_]`), which is stored in the second group for the name. It does the same with the third group for the duration. Note: To avoid unexpected string escaping in your regular expressions, it is recommended to use such as `r'raw-string'` instead of `'escaped-string'`. `Regex.matches` keeps only the elements that match the regular expression, returning the matched group. The argument `group` is set to `0` (the entire match) by default, but can be set to a group number like `3`, or to a named group like `'icon'`. `Regex.matches` starts to match the regular expression at the beginning of the string. To match until the end of the string, add `'$'` at the end of the regular expression. To start matching at any point instead of the beginning of the string, use . {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexMatches" show="regexmatches" >}} {{< /playground >}} `Regex.all_matches` keeps only the elements that match the regular expression, returning all groups as a list. The groups are returned in the order encountered in the regular expression, including `group 0` (the entire match) as the first group. `Regex.all_matches` starts to match the regular expression at the beginning of the string. To match until the end of the string, add `'$'` at the end of the regular expression. To start matching at any point instead of the beginning of the string, use . {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexAllMatches" show="regexall_matches" >}} {{< /playground >}} `Regex.matches_kv` keeps only the elements that match the regular expression, returning a key-value pair using the specified groups. The argument `keyGroup` is set to a group number like `3`, or to a named group like `'icon'`. The argument `valueGroup` is set to `0` (the entire match) by default, but can be set to a group number like `3`, or to a named group like `'icon'`. `Regex.matches_kv` starts to match the regular expression at the beginning of the string. To match until the end of the string, add `'$'` at the end of the regular

To start matching at any point instead of the beginning of the string, use . {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexMatchesKV" show="regexmatches_kv" >}} {{< /playground >}} `Regex.find` keeps only the elements that match the regular expression, returning the matched group. The argument `group` is set to `0` (the entire match) by default, but can be set to a group number like `3`, or to a named group like `'icon'`. `Regex.find` matches the first occurrence of the regular expression in the string. To start matching at the beginning, add `'^'` at the beginning of the regular expression. To match until the end of the string, add `'$'` at the end of the regular expression. If you need to match from the start only, consider using . {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexFind" show="regexfind" >}} {{< /playground >}} `Regex.find_all` returns a list of all the matches of the regular expression, returning the matched group. The argument `group` is set to `0` by default, but can be set to a group number like `3`, to a named group like `'icon'`, or to `Regex.ALL` to return all groups. The argument `outputEmpty` is set to `True` by default, but can be set to `False` to skip elements where no matches were found. `Regex.find_all` matches the regular expression anywhere it is found in the string. To start matching at the beginning, add `'^'` at the start of the regular expression. To match until the end of the string, add `'$'` at the end of the regular expression. If you need to match all groups from the start only, consider using . {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexFindAll" show="regexfind_all" >}} {{< /playground >}} `Regex.find_kv` returns a list of all the matches of the regular expression, returning a key-value pair using the specified groups. The argument `keyGroup` is set to a group number like `3`, or to a named group like `'icon'`. The argument `valueGroup` is set to `0` (the entire match) by default, but can be set to a group number like `3`, or to a named group like `'icon'`. `Regex.find_kv` matches the first occurrence of the regular expression in the string. To start matching at the beginning, add `'^'` at the beginning of the regular expression. To match until the end of the string, add `'$'` at the end of the regular expression. If you need to match as key-value pairs from the start only, consider using . {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexFindKV" show="regexfind_kv" >}} {{< /playground >}} `Regex.replace_all` returns the string with all the occurrences of the regular expression replaced by another string. You can also use on the `replacement`. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexReplaceAll" show="regexreplace_all" >}} {{< /playground >}} `Regex.replace_first` returns the string with the first occurrence of the regular expression replaced by another string. You can also use on the `replacement`. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexReplaceFirst" show="regexreplace_first" >}} {{< /playground >}} `Regex.split` returns the list of strings that were delimited by the specified regular expression. The argument `outputEmpty` is set to `False` by default, but can be set to `True` to keep empty items in the output list. {{< playground height="700px" >}} {{< playgroundsnippet language="py" path="SDKPYTHONRegexSplit" show="regexsplit" >}} {{< /playground >}} behaves the same as `Map`, but for each input it may produce zero or more outputs. applies a simple 1-to-1 mapping function over each element in the collection {{< button-pydoc path="apache_beam.transforms.util" class="Regex" >}}