Project: Beam (App Definition and Development / Streaming & Messaging)
File: beam-2.24.0.md

title: "Apache Beam 2.24.0" date: 2020-09-18 00:00:01 -0800 categories: blog release authors: danoliveira <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> We are happy to present the new 2.24.0 release of Apache Beam. This release includes both improvements and new functionality. See the for this release.<!--more--> For more information on changes in 2.24.0, check out the . Apache Beam 2.24.0 is the last release with Python 2 and Python 3.5 support. New overloads for BigtableIO.Read.withKeyRange() and BigtableIO.Read.withRowFilter() methods that take ValueProvider as a parameter (Java) (). The WriteToBigQuery transform (Python) in Dataflow Batch no longer relies on BigQuerySink by default. It relies on a new, fully-featured transform based on file loads into BigQuery. To revert the behavior to the old implementation, you may use `--experiments=uselegacybq_sink`. Add cross-language support to Java's JdbcIO, now available in the Python module `apache_beam.io.jdbc` (, ). Add support of AWS SDK v2 for KinesisIO.Read (Java) (). Add streaming support to SnowflakeIO in Java SDK () Support reading and writing to Google Healthcare DICOM APIs in Python SDK () Add dispositions for SnowflakeIO.write () Add cross-language support to SnowflakeIO.Read now available in the Python module `apache_beam.io.external.snowflake` (). Shared library for simplifying management of large shared objects added to Python SDK. Example use case is sharing a large TF model object across threads (). Dataflow streaming timers are not strictly time ordered when set earlier mid-bundle (). OnTimerContext should not create a new one when processing each element/timer in FnApiDoFnRunner () Key should be available in @OnTimer methods (Spark Runner) () WriteToBigQuery transforms now require a GCS location to be provided through either customgcstemp_location in the constructor of WriteToBigQuery or the fallback option --templocation, or pass method="STREAMINGINSERTS" to WriteToBigQuery (). Python SDK now understands `typing.FrozenSet` type hints, which are not interchangeable with `typing.Set`. You may need to update your pipelines if type checking fails. () Default compressor change breaks dataflow python streaming job update compatibility. Please use python SDK version <= 2.23.0 or > 2.25.0 if job update is critical.() According to git shortlog, the following people contributed to the 2.24.0 release. Thank you to all contributors! adesormi, Ahmet Altay, Alex Amato, Alexey Romanenko, Andrew Pilloud, Ashwin Ramaswami, Borzoo, Boyuan Zhang, Brian Hulette, Brian M, Bu Sun Kim, Chamikara Jayalath, Colm O hEigeartaigh, Corvin Deboeser, Damian Gadomski, Damon Douglas, Daniel Oliveira, Dariusz Aniszewski, davidak09, David Cavazos, David Moravek, David Yan, dhodun, Doug Roeper, Emil Hessman, Emily Ye, Etienne Chauchot, Etta Rapp, Eugene Kirpichov, fuyuwei, Gleb Kanterov, Harrison Green, Heejong Lee, Henry Suryawirawan, InigoSJ, Ismal Meja, Israel Herraiz, Jacob Ferriero, Jan Lukavsk, Jayendra, jfarr, jhnmora000, Jiadai Xia, JIahao wu, Jie Fan, Jiyong Jung, Julius Almeida, Kamil Gauszka, Kamil Wasilewski, Kasia Kucharczyk, Kenneth Knowles, Kevin Puthusseri, Kyle Weaver, ukasz Gajowy, Luke Cwik, Mark-Zeng, Maximilian Michels, Michal Walenia, Niel Markwick, Ning Kang, Pablo Estrada, pawel.urbanowicz, Piotr Szuberski, Rafi Kamal, rarokni, Rehman Murad Ali, Reuben van Ammers, Reuven Lax, Ricardo Bordon, Robert Bradshaw, Robert Burke, Robin Qiu, Rui Wang, Saavan Nanavati, sabhyankar, Sam Rohde, Scott Lukas, Siddhartha Thota, Simone Primarosa, Sawomir Andrian, Steve Niemitz, Tobiasz Kdzierski, Tomo Suzuki, Tyson Hamilton, Udi Meiri, Valentyn Tymofieiev, viktorjonsson, Xinyu Liu, Yichi Zhang, Yixing Zhang, yoshiki.obata, Yueyang Qiu, zijiesong