Project: Beam (App Definition and Development / Streaming & Messaging)
File: java-dependencies.md

type: languages title: "Java SDK dependencies" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> The Apache Beam SDKs depend on common third-party components. These components import additional dependencies. You need to manage your dependencies for the following reasons: Dependencies might have version collisions or incompatible classes and libraries. Some libraries are not forward compatible. When you use these packages in your code, you might need to pin to the appropriate versions so that those versions are used when you run your pipeline. When problems occur with dependencies, you might see unexpected behavior in the service, including errors such as `NoClassDefFoundError`, `NoSuchMethodError`, `NoSuchFieldError`, or `FATAL ERROR in native method`. This page explains how to view the dependencies that your SDK is using and how to manage your dependencies to avoid issues. To view your dependencies, either use the `BeamModulePlugin.groovy` file or retrieve the list by creating a new project through Maven and resolving the dependencies. The `BeamModulePlugin.groovy` file in the Beam repository lists compile and runtime dependencies for your Beam SDK version. Use the following link to open the `BeamModulePlugin.groovy` file. ``` https://raw.githubusercontent.com/apache/beam/vBEAM_VERSION/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy ``` <p class="paragraph-wrap">Replace <em>BEAM_VERSION</em> with the SDK version that you're using. The following example provides the dependencies for the {{< param releaselatest >}} release: <a href="https://raw.githubusercontent.com/apache/beam/v{{< param releaselatest >}}/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy" target="blank" rel="noopener noreferrer">https://raw.githubusercontent.com/apache/beam/v{{< param releaselatest >}}/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy</a>.</p> Under `project.ext.library`, review the list of dependencies. Some dependencies in the list use version variables, such as `googlecloudbigdataoss_version`. These variables are defined before the `project.ext.library` map definition. You can retrieve the list of dependencies by creating a new project through Maven and then resolving the

In your terminal or command line, use the following command to define the Beam SDK and Java versions for the new project. ``` export BEAMVERSION={{< param releaselatest >}} export JAVA_VERSION=11 ``` Create the project. ``` mvn archetype:generate \ -DinteractiveMode=false \ -DarchetypeGroupId=org.apache.beam \ -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-starter \ -DarchetypeVersion=$BEAM_VERSION \ -DtargetPlatform=$JAVA_VERSION \ -DartifactId=check-pipeline-dependencies \ -DgroupId=org.apache.beam.samples ``` Change to the new project directory. ``` cd check-pipeline-dependencies ``` Resolve and list the dependencies. ``` mvn dependency:resolve && mvn -o dependency:list ``` To simplify dependency management, Beam provides artifacts that help dependency management tools select compatible combinations. When you import Apache Beam, using a Bill of Material artifact is recommended. When a project import contains unspecified or ambiguous dependencies, the BOM provides the information that the SDK needs to use the correct dependency version. Apache Beam provides two BOMs: `beam-sdks-java-bom`: manages Apache Beam dependencies, which allows you to specify the version only one time `beam-sdks-java-io-google-cloud-platform-bom`: manages Apache Beam, Google Cloud, and third-party dependencies Because errors are more likely to occur when you use third-party dependencies, the `beam-sdks-java-io-google-cloud-platform-bom` BOM is recommended. To use a BOM, import the BOM into your Maven or Gradle dependency configuration. For example, to use `beam-sdks-java-io-google-cloud-platform-bom`, make the following changes in the `pom.xml` file of your SDK artifact. In the following examples, replace BEAMVERSION_ with the appropriate Apache Beam SDK version. Maven ```xml <dependencyManagement> <dependencies> <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-sdks-java-google-cloud-platform-bom</artifactId> <version>BEAM_VERSION</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> ``` Gradle ``` dependencies { implementation(platform("org.apache.beam:beam-sdks-java-google-cloud-platform-bom:BEAM_VERSION")) } ``` After you import the BOM, you can remove specific version pinning from your dependencies. For example, you can remove the versions associated with `org.apache.beam`, `io.grpc`, and `com.google.cloud`, including `libraries-bom`. Because the dependencies aren't automatically imported by the BOM, don't remove them entirely. Keep the dependency without specifying a version. The following example shows dependencies without versions in Maven: ```xml <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-sdks-java-core</artifactId> </dependency> ``` The following example shows dependencies without versions in Gradle: ``` implementation("org.apache.beam:beam-sdks-java-core") ``` To see a full list of dependency versions that are managed by a specific BOM, use the Maven tool `help:effective-pom` by running the following command. Replace BEAMVERSION_ with the appropriate Apache Beam SDK version. ```shell mvn help:effective-pom -f ~/.m2/repository/org/apache/beam/beam-sdks-java-google-cloud-platform-bom/BEAMVERSION/beam-sdks-java-google-cloud-platform-bom-BEAMVERSION.pom ``` in the Maven repository.