Bug: Revisit codes in Scala tests
Status: Open | Priority: Major | Resolution: 
Created: 26/Dec/21 07:14
Affects Version/s: 3.3.0

{code}
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala:  // TODO(SPARK-9603): Building a package is flaky on Jenkins Maven builds.
core/src/test/scala/org/apache/spark/deploy/SparkSubmitTestUtils.scala:      // This test suite has some weird behaviors when executed on Jenkins:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitTestUtils.scala:      // 1. Sometimes it gets extremely slow out of unknown reason on Jenkins.  Here we add a
core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala:      // is only 2, while on Jenkins it's 32. For this specific test, 2 available processors, which
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDontFailOnDataLossSuite.scala: * when running on a slow Jenkins machine) before records start to be removed. To make sure a test
project/SparkBuild.scala:      // with Jenkins flakiness.
repl/src/test/scala/org/apache/spark/repl/SparkShellSuite.scala:      // This test suite sometimes gets extremely slow out of unknown reason on Jenkins.  Here we
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala:    // TODO: Why fs.getContentSummary returns wrong size on Jenkins?
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala:    // Seems fs.getContentSummary returns wrong table size on Jenkins. So we use
sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala:    // the last dot due to a bug in SBT. This makes easier to debug via Jenkins test result
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreIntegrationSuite.scala:          // Should emit new progresses every 10 ms, but we could be facing a slow Jenkins
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryStatusAndProgressSuite.scala:        // Should emit new progresses every 10 ms, but we could be facing a slow Jenkins
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala:      // This test suite sometimes gets extremely slow out of unknown reason on Jenkins.  Here we
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala:          // started at a time, which is not Jenkins friendly.
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientBuilder.scala:  // In order to speed up test execution during development or in Jenkins, you can specify the path
sql/hive/src/test/scala/org/apache/spark/sql/sources/ParquetHadoopFsRelationSuite.scala:  // more cores, the issue can be reproduced steadily.  Fortunately our Jenkins builder meets this
streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala:      // If Jenkins is slow, we may not have a chance to run many threads simultaneously. Having
core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala:      // This is to avoid any potential flakiness in the test because of large pauses in jenkins
sql/core/src/test/java/test/org/apache/spark/sql/JavaDataFrameSuite.java:      // (e.g., /home/jenkins/workspace/SparkPullRequestBuilder@2)
sql/core/src/test/resources/sql-tests/inputs/datetime-parsing-invalid.sql:-- in java 8 this case is invalid, but valid in java 11, disabled for jenkins
sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala:    // TODO: Figure out why StringType doesn't work on jenkins.
sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala:    // Weird DDL differences result in failures on jenkins.
project/SparkBuild.scala:      // with Jenkins flakiness.
{code}