Bug: Fix Scala 2.13 mima check after SPARK-44705 merged
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Aug/23 03:59
Affects Version/s: 4.0.0

{code:java}
[error] spark-core: Failed binary compatibility check against org.apache.spark:spark-core_2.13:3.4.0! Found 1 potential problems (filtered 4019)
8107[error]  * method this(org.apache.spark.api.python.BasePythonRunner,java.io.DataInputStream,org.apache.spark.api.python.BasePythonRunner#WriterThread,Long,org.apache.spark.SparkEnv,java.net.Socket,scala.Option,java.util.concurrent.atomic.AtomicBoolean,org.apache.spark.TaskContext)Unit in class org.apache.spark.api.python.BasePythonRunner#ReaderIterator's type is different in current version, where it is (org.apache.spark.api.python.BasePythonRunner,java.io.DataInputStream,org.apache.spark.api.python.BasePythonRunner#Writer,Long,org.apache.spark.SparkEnv,org.apache.spark.api.python.PythonWorker,scala.Option,java.util.concurrent.atomic.AtomicBoolean,org.apache.spark.TaskContext)Unit instead of (org.apache.spark.api.python.BasePythonRunner,java.io.DataInputStream,org.apache.spark.api.python.BasePythonRunner#WriterThread,Long,org.apache.spark.SparkEnv,java.net.Socket,scala.Option,java.util.concurrent.atomic.AtomicBoolean,org.apache.spark.TaskContext)Unit
8108[error]    filter with: ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.api.python.BasePythonRunner#ReaderIterator.this")
8109[error] java.lang.RuntimeException: Failed binary compatibility check against org.apache.spark:spark-core_2.13:3.4.0! Found 1 potential problems (filtered 4019)
8110[error] 	at scala.sys.package$.error(package.scala:30)
8111[error] 	at com.typesafe.tools.mima.plugin.SbtMima$.reportModuleErrors(SbtMima.scala:89)
8112[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$2(MimaPlugin.scala:36)
8113[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$2$adapted(MimaPlugin.scala:26)
8114[error] 	at scala.collection.Iterator.foreach(Iterator.scala:943)
8115[error] 	at scala.collection.Iterator.foreach$(Iterator.scala:943)
8116[error] 	at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
8117[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$1(MimaPlugin.scala:26)
8118[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$1$adapted(MimaPlugin.scala:25)
8119[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
8120[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63)
8121[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:69)
8122[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
8123[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
8124[error] 	at sbt.Execute.work(Execute.scala:292)
8125[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
8126[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
8127[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
8128[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
8129[error] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
8130[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
8131[error] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
8132[error] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
8133[error] 	at java.lang.Thread.run(Thread.java:750)
8134[error] (core / mimaReportBinaryIssues) Failed binary compatibility check against org.apache.spark:spark-core_2.13:3.4.0! Found 1 potential problems (filtered 4019)
8135[error] Total time: 162 s (02:42), completed Aug 13, 2023 7:28:17 PM
8136
8137Error:  running /home/runner/work/spark/spark/dev/mima -Phadoop-3 -Pscala-2.13 -Pconnect -Pmesos -Pkubernetes -Phadoop-cloud -Pspark-ganglia-lgpl -Pkinesis-asl -Phive -Pdocker-integration-tests -Pyarn -Pvolcano -Phive-thriftserver ; received return code 1
8138Error: Process completed with exit code 17. {code}