Project: Beam (App Definition and Development / Streaming & Messaging)
File: beam-2.17.0.md

title: "Apache Beam 2.17.0" date: 2020-01-06 00:00:01 -0800 categories: blog release aliases: /blog/2020/01/06/beam-2.17.0.html authors: ardagan <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> We are happy to present the new 2.17.0 release of Beam. This release includes both improvements and new functionality. Users of the MongoDbIO connector are encouraged to upgrade to this release to address a . See the for this release.<!--more--> For more information on changes in 2.17.0, check out the . - Drop support for Flink 1.5 and 1.6 - Migrate SnsIO to AWS SDK for Java 2 Improved usability for portable Flink Runner - Optionally bundle multiple pipelines into a single Flink jar. - Allow submission of Flink UberJar directly to flink cluster. - Flink native job submission for portable pipelines. - Flink portable pipeline jars do not need to stage artifacts remotely. - Add Flink 1.9 build target and Make FlinkRunner compatible with Flink 1.9. - Add ability to read parquet files into PCollection of pyarrow.Table. - Make BooleanCoder a standard coder. - Add withDataSourceConfiguration() method in JdbcIO.ReadRows class. - Implement cross-bundle state caching. - Add handling of DynamicMessage in ProtoCoder. - Update RestrictionTracker within Python to not be required to be thread safe. - Added AvroTableProvider to Beam SQL. - Improve documentation on BigQueryIO. - Add exception handling to Json transforms in Java SDK. - Improve estimation of data byte size reading from source in ElasticsearchIO. - Support passing in arbitrary KV pairs to sdk worker via external environment config. - Default to LOOPBACK mode for local flink (spark, ...) runner. - JdbcIO should support setConnectionInitSqls in its DataSource. - Add HllCount to Java transform catalog. - Disallow self-signed certificates by default in ElasticsearchIO. - Upgrade ZetaSQL to 2019.09.1. - Upgrade pyarrow version bounds: 0.15.1<= to <0.16.0. - Upgrade com.rabbitmq:amqp-client to 5.7.3. - Upgrade PyYAML version bounds: 3.12<= to <6.0.0. [BEAM-8819] - AvroCoder for SpecificRecords is not serialized correctly since 2.13.0 Various bug fixes and performance improvements. Apache Nemo runner broken due to backwards incompatible change since 2.16.0. According to git shortlog, the following people contributed to the 2.17.0 release. Thank you to all contributors! Ahmet Altay, Alan Myrvold, Alexey Romanenko, Andre-Philippe Paquet, Andrew Pilloud, angulartist, Ankit Jhalaria, Ankur Goenka, Anton Kedin, Aryan Naraghi, Aurlien Geron, B M VISHWAS, Bartok Jozsef, Boyuan Zhang, Brian Hulette, Cerny Ondrej, Chad Dombrova, Chamikara Jayalath, ChethanU, cmach, Colm O hEigeartaigh, Cyrus Maden, Daniel Oliveira, Daniel Robert, Dante, David Cavazos, David Moravek, David Yan, Enrico Canzonieri, Etienne Chauchot, gxercavins, Hai Lu, Hannah Jiang, Ian Lance Taylor, Ismal Meja, Israel Herraiz, James Wen, Jan Lukavsk, Jean-Baptiste Onofr, Jeff Klukas, jesusrv1103, Jofre, Kai Jiang, Kamil Wasilewski, Kasia Kucharczyk, Kenneth Knowles, Kirill Kozlov, kirillkozlov, Kohki YAMAGIWA, Kyle Weaver, Leonardo Alves Miguel, lloigor, lostluck, Luis Enrique Ortz Ramirez, Luke Cwik, Mark Liu, Maximilian Michels, Michal Walenia, Mikhail Gryzykhin, mrociorg, Nicolas Delsaux, Ning Kang, NING KANG, Pablo Estrada, pabloem, Piotr Szczepanik, rahul8383, Rakesh Kumar, Renat Nasyrov, Reuven Lax, Robert Bradshaw, Robert Burke, Rui Wang, Ruslan Altynnikov, Ryan Skraba, Salman Raza, Saul Chavez, Sebastian Jambor, sunjincheng121, Tatu Saloranta, tchiarato, Thomas Weise, Tomo Suzuki, Tudor Marian, tvalentyn, Udi Meiri, Valentyn Tymofieiev, Viola Lyu, Vishwas, Yichi Zhang, Yifan Zou, Yueyang Qiu, ukasz Gajowy