Project: Beam (App Definition and Development / Streaming & Messaging)
File: managing-beam-dependencies-in-java.md

title: "Managing Beam dependencies in Java" date: 2023-06-23 9:00:00 -0700 categories: blog authors: bvolpato <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Managing your Java dependencies can be challenging, and if not done correctly, it may cause a variety of problems, as incompatibilities may arise when using specific and previously untested combinations. To make that process easier, Beam now provides artifacts that will help dependency management tools to select compatible combinations. We hope this will make it easier for you to use Apache Beam, and have a simpler transition when upgrading to newer versions. <!--more--> When bringing incompatible classes and libraries, the code is susceptible to errors such as `NoClassDefFoundError`, `NoSuchMethodError`, `NoSuchFieldError`, `FATAL ERROR in native method`. When importing Apache Beam, the recommended way is to use Bill of Materials (BOMs). The way BOMs work is by providing hints to the dependency management resolution tool, so when a project imports unspecified or ambiguous dependencies, it will know what version to use. There are currently two BOMs provided by Beam: `beam-sdks-java-bom`, which manages what dependencies of Beam will be used, so you can specify the version only once. `beam-sdks-java-io-google-cloud-platform-bom`, a more comprehensive list, which manages Beam, along with GCP client and third-party dependencies. Since errors are more likely to arise when using third-party dependencies, that's the one that is recommended to use to minimize any conflicts. In order to use BOM, the artifact has to be imported to your Maven or Gradle dependency configurations. For example, to use `beam-sdks-java-io-google-cloud-platform-bom`, the following changes have to be done (and make sure that BEAMVERSION_ is replaced by a valid version): Maven ```xml <dependencyManagement> <dependencies> <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-sdks-java-google-cloud-platform-bom</artifactId> <version>BEAM_VERSION</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> ``` Gradle ``` dependencies { implementation(platform("org.apache.beam:beam-sdks-java-google-cloud-platform-bom:BEAM_VERSION")) } ``` After importing the BOM, specific version pinning of dependencies, for example, anything for `org.apache.beam`, `io.grpc`, `com.google.cloud` ( including `libraries-bom`) may be removed. Do not entirely remove the dependencies, as they are not automatically imported by the BOM. It is important to keep the dependency without specifying a version. For example, in Maven: ```xml <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-sdks-java-core</artifactId> </dependency> ``` Or Gradle: ``` implementation("org.apache.beam:beam-sdks-java-core") ``` For a full list of dependency versions that are managed by a specific BOM, the Maven tool `help:effective-pom` can be used. For example: ```shell mvn help:effective-pom -f ~/.m2/repository/org/apache/beam/beam-sdks-java-google-cloud-platform-bom/BEAMVERSION/beam-sdks-java-google-cloud-platform-bom-BEAMVERSION.pom ``` The third-party website can also be used to display such version information. We hope you find this useful. and contributions are always welcome! So feel free to create a GitHub issue, or open a Pull Request if you encounter any problem when using those artifacts.