Bug: Jdbc driver class not found  when enable hive and call  jdbc action function multiple times
Status: Open | Priority: Major | Resolution: 
Created: 07/Apr/22 09:29
Affects Version/s: 3.1.1

Hello, the spark code is:
{code:java}
// Some comments here
def main(args : Array[String]): Unit = {
    val spark = SparkSession.builder
      .appName("TestCKJdbc")
      .enableHiveSupport()
      .getOrCreate()

    spark.read
      .format("jdbc")
      .option("driver","ru.yandex.clickhouse.ClickHouseDriver")
      .option("url", "jdbc:clickhouse://clickhouse-server-svc.admin.svc.cluster.local:8123/aaa")
      .option("dbtable", "A")
      .option("user", "default")
      .option("password", "abc")
      .load()
      .show()

    spark.read
      .format("jdbc")
      .option("driver","ru.yandex.clickhouse.ClickHouseDriver")
      .option("url", "jdbc:clickhouse://clickhouse-server-svc.admin.svc.cluster.local:8123/aaa")
      .option("dbtable", "A")
      .option("user", "default")
      .option("password", "abc")
      .load()
      .show()

    spark.stop()
  }
{code}
when I submit it to k8s, it will get error:
{code:java}
22/04/07 09:12:08 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0) (10.233.67.129, executor 1, partition 0, PROCESS_LOCAL, 4318 bytes) taskResourceAssignments Map()
22/04/07 09:12:09 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.233.67.129:41816 (size: 5.1 KiB, free: 413.9 MiB)
22/04/07 09:12:15 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 6596 ms on 10.233.67.129 (executor 1) (1/1)
22/04/07 09:12:15 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 
22/04/07 09:12:15 INFO DAGScheduler: ResultStage 0 (show at App.scala:24) finished in 8.524 s
22/04/07 09:12:15 INFO DAGScheduler: Job 0 is finished. Cancelling potential speculative or zombie tasks for this job
22/04/07 09:12:15 INFO TaskSchedulerImpl: Killing all running tasks in stage 0: Stage finished
22/04/07 09:12:15 INFO DAGScheduler: Job 0 finished: show at App.scala:24, took 8.611591 s
22/04/07 09:12:15 INFO CodeGenerator: Code generated in 47.462472 ms
+---+---------+---------+
| id|discrete1|discrete2|
+---+---------+---------+
|  1|        A|        a|
|  1|        B|        b|
|  1|        A|        c|
|  2|        C|        a|
|  1|        A|        a|
+---+---------+---------+22/04/07 09:12:15 INFO SparkUI: Stopped Spark web UI at http://spark-7f8c3c80034ac34e-driver-svc.spark-operator.svc:4040
22/04/07 09:12:15 INFO KubernetesClusterSchedulerBackend: Shutting down all executors
22/04/07 09:12:15 INFO KubernetesClusterSchedulerBackend$KubernetesDriverEndpoint: Asking each executor to shut down
22/04/07 09:12:15 WARN ExecutorPodsWatchSnapshotSource: Kubernetes client has been closed (this is expected if the application is shutting down.)
22/04/07 09:12:15 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
22/04/07 09:12:15 INFO MemoryStore: MemoryStore cleared
22/04/07 09:12:15 INFO BlockManager: BlockManager stopped
22/04/07 09:12:15 INFO BlockManagerMaster: BlockManagerMaster stopped
22/04/07 09:12:15 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
22/04/07 09:12:15 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.lang.ClassNotFoundException: ru.yandex.clickhouse.ClickHouseDriver
    at java.base/java.net.URLClassLoader.findClass(Unknown Source)
    at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
    at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
    at org.apache.spark.sql.execution.datasources.jdbc.DriverRegistry$.register(DriverRegistry.scala:46)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.$anonfun$driverClass$1(JDBCOptions.scala:102)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.$anonfun$driverClass$1$adapted(JDBCOptions.scala:102)
    at scala.Option.foreach(Option.scala:407)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.<init>(JDBCOptions.scala:102)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.<init>(JDBCOptions.scala:38)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:32)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:354)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:326)
    at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:308)
    at scala.Option.getOrElse(Option.scala:189)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:308)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:226)
    at org.example.App$.main(App.scala:34)
    at org.example.App.main(App.scala)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.base/java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
22/04/07 09:12:15 INFO ShutdownHookManager: Shutdown hook called
22/04/07 09:12:15 INFO ShutdownHookManager: Deleting directory /tmp/spark-77261146-995b-42e6-b5e8-eb7cbf8eda49
22/04/07 09:12:15 INFO ShutdownHookManager: Deleting directory /var/data/spark-5afa86cb-67c6-493a-90f5-ea1c23089c9f/spark-d1c96f2e-c40c-4942-bd99-0451d23159b3
22/04/07 09:12:15 INFO AuditProviderFactory: ==> JVMShutdownHook.run()
22/04/07 09:12:15 INFO AuditProviderFactory: JVMShutdownHook: Signalling async audit cleanup to start.
22/04/07 09:12:15 INFO AuditProviderFactory: JVMShutdownHook: Waiting up to 30 seconds for audit cleanup to finish.
22/04/07 09:12:15 INFO AuditProviderFactory: RangerAsyncAuditCleanup: Starting cleanup
22/04/07 09:12:15 INFO AuditAsyncQueue: Stop called. name=hiveCLI.async
22/04/07 09:12:15 INFO AuditAsyncQueue: Interrupting consumerThread. name=hiveCLI.async, consumer=hiveCLI.async.batch
22/04/07 09:12:15 INFO AuditProviderFactory: RangerAsyncAuditCleanup: Done cleanup {code}
**but, if del ".enableHiveSupport()" code, it will run success.*
like:
{code:java}
def main(args : Array[String]): Unit = {
    val spark = SparkSession.builder
      .appName("TestCKJdbc")
      //.enableHiveSupport()
      .getOrCreate()

    spark.read
      .format("jdbc")
      .option("driver","ru.yandex.clickhouse.ClickHouseDriver")
      .option("url", "jdbc:clickhouse://clickhouse-server-svc.admin.svc.cluster.local:8123/abc")
      .option("dbtable", "A")
      .option("user", "default")
      .option("password", "aaa")
      .load()
      .show()

    spark.read
      .format("jdbc")
      .option("driver","ru.yandex.clickhouse.ClickHouseDriver")
      .option("url", "jdbc:clickhouse://clickhouse-server-svc.admin.svc.cluster.local:8123/abc")
      .option("dbtable", "A")
      .option("user", "default")
      .option("password", "aaa")
      .load()
      .show()

    spark.stop()
  }
{code}

It will happen only when enableHive and call multiple jdbc action function.