Project: Beam (App Definition and Development / Streaming & Messaging)
File: beam-2.54.0.md

title: "Apache Beam 2.54.0" date: 2024-02-14 09:00:00 -0400 categories: blog release authors: lostluck <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> We are happy to present the new 2.54.0 release of Beam. This release includes both improvements and new functionality. See the for this release. <!--more--> For more information on changes in 2.54.0, check out the . along with GCP BigTable handler added to Python SDK (). Beam Java Batch pipelines run on Google Cloud Dataflow will default to the Portable Runner (v2) starting with this version. (All other languages are already on Runner V2.) See for how to enable or disable it intentionally. Added support for writing to BigQuery dynamic destinations with Python's Storage Write API () Adding support for Tuples DataType in ClickHouse (Java) (). Added support for handling bad records to FileIO, TextIO, AvroIO (). Added support for handling bad records to BigtableIO (). along with GCP BigTable handler added to Python SDK (). N/A N/A Fixed a memory leak affecting some Go SDK since 2.46.0. () N/A Some Python pipelines that run with 2.52.0-2.54.0 SDKs and use large materialized side inputs might be affected by a performance regression. To restore the prior behavior on these SDK versions, supply the `--maxcachememoryusagemb=0` pipeline option. (). Python pipelines that run with 2.53.0-2.54.0 SDKs and perform file operations on GCS might be affected by excess HTTP requests. This could lead to a performance regression or a permission issue. () In Python pipelines, when shutting down inactive bundle processors, shutdown logic can overaggressively hold the lock, blocking acceptance of new work. Symptoms of this issue include slowness or stuckness in long-running jobs. Fixed in 2.56.0 (). For the most up to date list of known issues, see https://github.com/apache/beam/blob/master/CHANGES.md According to git shortlog, the following people contributed to the 2.54.0 release. Thank you to all contributors! Ahmed Abualsaud Alexey Romanenko Anand Inguva Andrew Crites Arun Pandian Bruno Volpato caneff Chamikara Jayalath Changyu Li Cheskel Twersky Claire McGinty clmccart Damon Danny McCormick dependabot[bot] Edward Cheng Ferran Fernndez Garrido Hai Joey Tran hugo-syn Issac Jack McCluskey Jan Lukavsk JayajP Jeffrey Kinard Jerry Wang Jing Joey Tran johnjcasey Kenneth Knowles Knut Olav Lite liferoad Marc Mark Zitnik martin trieu Mattie Fu Naireen Hussain Neeraj Bansal Niel Markwick Oleh Borysevych pablo rodriguez defino Rebecca Szper Ritesh Ghorse Robert Bradshaw Robert Burke Sam Whittle Shunping Huang Svetak Sundhar S. Veyri Talat UYARER tvalentyn Vlado Djerek Yi Hu Zechen Jian