Project: Beam (App Definition and Development / Streaming & Messaging)
File: beam-2.33.0.md

title: "Apache Beam 2.33.0" date: 2021-10-07 00:00:01 -0800 categories: blog release authors: udim <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> We are happy to present the new 2.33.0 release of Beam. This release includes both improvements and new functionality. See the for this release. <!--more--> For more information on changes in 2.33.0, check out the [detailed release notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404). Go SDK is no longer experimental, and is officially part of the Beam release process. Matching Go SDK containers are published on release. Batch usage is well supported, and tested on Flink, Spark, and the Python Portable Runner. SDK Tests are also run against Google Cloud Dataflow, but this doesn't indicate reciprocal support. The SDK supports Splittable DoFns, Cross Language transforms, and most Beam Model basics. Go Modules are now used for dependency management. This is a breaking change, see Breaking Changes for resolution. Easier path to contribute to the Go SDK, no need to set up a GO\_PATH. Minimum Go version is now Go v1.16 See the announcement blogpost for full information once published. <!-- {$TOPICS e.g.:} Support for X source added (Java) (). {$TOPICS} --> Projection pushdown in SchemaIO (). Upgrade Flink runner to Flink versions 1.13.2, 1.12.5 and 1.11.4 (). Since release 2.30.0, "The AvroCoder changes for BEAM-2303 \). Python GBK will stop supporting unbounded PCollections that have global windowing and a default trigger in Beam

This can be overriden with `--allowunsafetriggers`. (). Python GBK will start requiring safe triggers or the `--allowunsafetriggers` flag starting with Beam 2.34. (). UnsupportedOperationException when reading from BigQuery tables and converting TableRows to Beam Rows (Java) (). SDFBoundedSourceReader behaves much slower compared with the original behavior of BoundedSource (Python) (). ORDER BY column not in SELECT crashes (ZetaSQL) (). Spark 2.x users will need to update Spark's Jackson runtime dependencies (`spark.jackson.version`) to at least version 2.9.2, due to Beam updating its dependencies. See a full list of open this version. Go SDK jobs may produce "Failed to deduce Step from MonitoringInfo" messages following successful job execution. The messages are benign and don't indicate job failure. These are due to not yet handling PCollection metrics. Large Java BigQueryIO writes with the FILE_LOADS method will fail in batch mode (specifically, when copy jobs are used). This results in the error message: `IllegalArgumentException: Attempting to access unknown side input`. Please upgrade to a newer version (> 2.34.0) or use another write method (e.g. `STORAGEWRITEAPI`). According to git shortlog, the following people contributed to the 2.33.0 release. Thank you to all contributors! Ahmet Altay, Alex Amato, Alexey Romanenko, Andreas Bergmeier, Andres Rodriguez, Andrew Pilloud, Andy Xu, Ankur Goenka, anthonyqzhu, Benjamin Gonzalez, Bhupinder Sindhwani, Chamikara Jayalath, Claire McGinty, Daniel Mateus Pires, Daniel Oliveira, David Huntsperger, Dylan Hercher, emily, Emily Ye, Etienne Chauchot, Eugene Nikolaiev, Heejong Lee, iindyk, Iigo San Jose Visiers, Ismal Meja, Jack McCluskey, Jan Lukavsk, Jeff Ruane, Jeremy Lewi, KevinGG, Ke Wu, Kyle Weaver, lostluck, Luke Cwik, Marwan Tammam, masahitojp, Mehdi Drissi, Minbo Bae, Ning Kang, Pablo Estrada, Pascal Gillet, Pawas Chhokra, Reuven Lax, Ritesh Ghorse, Robert Bradshaw, Robert Burke, Rodrigo Benenson, Ryan Thompson, Saksham Gupta, Sam Rohde, Sam Whittle, Sayat, Sayat Satybaldiyev, Siyuan Chen, Slava Chernyak, Steve Niemitz, Steven Niemitz, tvalentyn, Tyson Hamilton, Udi Meiri, vachan-shetty, Venkatramani Rajgopal, Yichi Zhang, zhoufek