Project: Beam (App Definition and Development / Streaming & Messaging)
File: beam-2.51.0.md

title: "Apache Beam 2.51.0" date: 2023-10-11 09:00:00 -0400 categories: blog release authors: klk <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> We are happy to present the new 2.51.0 release of Beam. This release includes both improvements and new functionality. See the for this release. <!--more--> For more information on changes in 2.51.0, check out the . In Python, now supports loading many models in the same transform using a (). In Python, the now supports passing in inference_args. These will be passed through to the Vertex endpoint as parameters. Added support to run `mypy` on user pipelines () Removed fastjson library dependency for Beam SQL. Table property is changed to be based on jackson ObjectNode (Java) (). Removed TensorFlow from Beam Python container images . If you have been negatively affected by this change, please comment on . Removed the parameter `t reflect.Type` from `parquetio.Write`. The element type is derived from the input PCollection (Go) () Refactor BeamSqlSeekableTable.setUp adding a parameter joinSubsetType. Fixed exception chaining issue in GCS connector (Python) (). Fixed streaming inserts exception handling, GoogleAPICallErrors are now retried according to retry strategy and routed to failed rows where appropriate rather than causing a pipeline error (Python) (). Fixed a bug in Python SDK's cross-language Bigtable sink that mishandled records that don't have an explicit timestamp set: . Python containers updated, fixing , , , , , , and Used go 1.21.1 to build, fixing Python pipelines using BigQuery Storage Read API must pin `fastavro` dependency to 1.8.3 or earlier: According to git shortlog, the following people contributed to the 2.50.0 release. Thank you to all contributors! Adam Whitmore Ahmed Abualsaud Ahmet Altay Aleksandr Dudko Alexey Romanenko Anand Inguva Andrey Devyatkin Arvind Ram Arwin Tio BjornPrime Bruno Volpato Bulat Celeste Zeng Chamikara Jayalath Clay Johnson Damon Danny McCormick David Cavazos Dip Patel Hai Joey Tran Hao Xu Haruka Abe Jack Dingilian Jack McCluskey Jeff Kinard Jeffrey Kinard Joey Tran Johanna jeling Julien Tournay Kenneth Knowles Kerry Donny-Clark Mattie Fu Melissa Pashniak Michel Davit Moritz Mack Pranav Bhandari Rebecca Szper Reeba Qureshi Reuven Lax Ritesh Ghorse Robert Bradshaw Robert Burke Ruwann Ryan Tam Sam Rohde Sereana Seim Svetak Sundhar Tim Grein Udi Meiri Valentyn Tymofieiev Vitaly Terentyev Vlado Djerek Xinyu Liu Yi Hu Zbynek Konecny Zechen Jiang bzablocki caneff dependabot[bot] gDuperran gabry.wu johnjcasey kberezin-nshl kennknowles liferoad lostluck magicgoody martin trieu mosche olalamichelle tvalentyn xqhu ukasz Spyra