Bug: Upgrade protobuf-java from 2.5.0 to 3.16.1
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 27/Feb/22 12:39
Affects Version/s: 3.3.0

 [CVE-2021-22569|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-22569]

To do this upgrade I have done

external/kinesis-asl-assembly/pom.xml change line 65 to <version>3.16.1</version> 

pom.xml change line 124 to <protobuf.version>3.16.1</protobuf.version>

run 
./dev/test-dependencies.sh --replace-manifest 

witch change 
dev/deps/spark-deps-hadoop-2-hive-2.3 line 235 to protobuf-java/3.16.1//protobuf-java-3.16.1.jar

and 

dev/deps/spark-deps-hadoop-3-hive-2.3 to protobuf-java/3.16.1//protobuf-java-3.16.1.jar 

My branch [protobuf-java-from-2.5.0-to-3.16.1|https://github.com/bjornjorgensen/spark/tree/protobuf-java-from-2.5.0-to-3.16.1] is OK with testes, but  when I run 

./build/mvn -DskipTests clean package && ./build/mvn -e package
 

I get this error:

01:01:41.381 WARN org.apache.spark.sql.execution.datasources.orc.OrcColumnarBatchReaderSuite: 

===== POSSIBLE THREAD LEAK IN SUITE o.a.s.sql.execution.datasources.orc.OrcColumnarBatchReaderSuite, threads: rpc-boss-3348-1 (daemon=true), shuffle-boss-3351-1 (daemon=true) =====

Run completed in 1 hour, 7 minutes, 35 seconds.
Total number of tests run: 11260
Suites: completed 505, aborted 0
Tests: succeeded 11259, failed 1, canceled 5, ignored 57, pending 0
*** 1 TEST FAILED ***
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.3.0-SNAPSHOT:
[INFO] 
[INFO] Spark Project Parent POM ........................... SUCCESS [  3.396 s]
[INFO] Spark Project Tags ................................. SUCCESS [  7.374 s]
[INFO] Spark Project Sketch ............................... SUCCESS [  9.324 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  4.097 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 47.468 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 10.478 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  2.425 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  2.767 s]
[INFO] Spark Project Core ................................. SUCCESS [30:56 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 29.105 s]
[INFO] Spark Project GraphX ............................... SUCCESS [02:09 min]
[INFO] Spark Project Streaming ............................ SUCCESS [05:21 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [08:15 min]
[INFO] Spark Project SQL .................................. FAILURE [  01:11 h]
[INFO] Spark Project ML Library ........................... SKIPPED
[INFO] Spark Project Tools ................................ SKIPPED
[INFO] Spark Project Hive ................................. SKIPPED
[INFO] Spark Project REPL ................................. SKIPPED
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SKIPPED
[INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
[INFO] Spark Avro ......................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:00 h
[INFO] Finished at: 2022-02-27T01:01:44+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:2.0.2:test (test) on project spark-sql_2.12: There are test failures -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.scalatest:scalatest-maven-plugin:2.0.2:test (test) on project spark-sql_2.12: There are test failures
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:972)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures
    at org.scalatest.tools.maven.TestMojo.execute (TestMojo.java:109)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:972)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :spark-sql_2.12