Project: Beam (App Definition and Development / Streaming & Messaging)
File: beam-2.20.0.md

title: "Apache Beam 2.20.0" date: 2020-04-15 00:00:01 -0800 aliases: /blog/2020/03/01/beam-2.20.0.html /blog/2020/04/15/beam-2.20.0.html categories: blog release authors: amaliujia <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> We are happy to present the new 2.20.0 release of Beam. This release includes both improvements and new functionality. See the for this release.<!--more--> For more information on changes in 2.20.0, check out the . Python SDK: . (#10223). Adds support for Thrift encoded data via ThriftIO KafkaIO supports schema resolution using Confluent Schema Registry Support for Google Cloud Spanner. This is an experimental module for reading and writing data from Google Cloud Spanner Adds support for standard HDFS URLs (with server name) New AnnotateVideo & AnnotateVideoWithContext PTransform's that integrates GCP Video Intelligence functionality New AnnotateImage & AnnotateImageWithContext PTransform's for element-wise & batch image annotation using Google Cloud Vision API Added a PTransform for inspection and deidentification of text using Google Cloud DLP New AnnotateText PTransform that integrates Google Cloud Natural Language functionality ReadFromBigQuery now supports value providers for the query string Added ability to write to BigQuery via Avro file loads Direct runner for FnApi supports further parallelism Support for @RequiresTimeSortedInput in Flink and Spark Added support for dynamic timers Backwards incompatible change in ReadFromPubSub(topic=) in Python SpannerAccessor in Java is now package-private to reduce API surface ParquetIO hadoop dependency should be now provided by the users Docker images will be deployed to apache/beam repositories from 2.20 Fixed numpy operators in ApproximateQuantiles Fixed exception when running in IPython notebook Restructure Python pipeline construction to better follow the Runner API Fixed Flink uberjar job termination bug Fixed SyntaxError in process worker startup Various bug fixes and performance improvements. Python SDK ignores manually set PCollection tags Python SDK pre_optimize=all experiment may cause error Python SDK performance regression for reshuffle transform According to git shortlog, the following people contributed to the 2.20.0 release. Thank you to all contributors! Ahmet Altay, Alex Amato, Alexey Romanenko, Andrew Pilloud, Ankur Goenka, Anton Kedin, Boyuan Zhang, Brian Hulette, Brian Martin, Chamikara Jayalath , Charles Chen, Craig Chambers, Daniel Oliveira, David Moravek, David Rieber, Dustin Rhodes, Etienne Chauchot, Gleb Kanterov, Hai Lu, Heejong Lee , Ismal Meja, J Ross Thomson, Jan Lukavsk, Jason Kuster, Jean-Baptiste Onofr, Jeff Klukas, Joo Cabrita, Juan Rael, Juta, Kasia Kucharczyk , Kengo Seki, Kenneth Jung, Kenneth Knowles, Kyle Weaver, Kyle Winkelman, Lukas Drbal, Marek Simunek, Mark Liu, Maximilian Michels, Melissa Pashniak , Michael Luckey, Michal Walenia, Mike Pedersen, Mikhail Gryzykhin, Niel Markwick, Pablo Estrada, Pascal Gula, Rehman Murad Ali, Reuven Lax, Rob, Robbe Sneyders , Robert Bradshaw, Robert Burke, Rui Wang, Ruoyun Huang, Ryan Williams, Sam Rohde, Sam Whittle, Scott Wegner, Shoaib Zafar, Thomas Weise, Tianyang Hu, Tyler Akidau , Udi Meiri, Valentyn Tymofieiev, Xinyu Liu, XuMingmin, ttanay, tvalentyn, ukasz Gajowy