av9ash/GitBugs huggingface dataset:
Bug: JDBCRelation columnPartition function improperly determines stride size. Upper bound is skewed due to stride alignment.
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Mar/21 20:37
Affects Version/s: 3.2.0

Currently, in JDBCRelation (line 123), the stride size is calculated as follows:

val stride: Long = upperBound / numPartitions - lowerBound / numPartitions

 

Due to truncation happening on both divisions, the stride size can fall short of what it should be. This can lead to a big difference between the provided upper bound and the actual start of the last partition.

I propose this formula, as it is much more accurate and leads to better distribution:

val stride = (upperBound / numPartitions.toFloat - lowerBound / numPartitions.toFloat).toLong

 

An example (using a date column):

Say you're creating 1,000 partitions. If you provide a lower bound of 1927-04-05 (this gets translated to -15611), and an upper bound of 2020-10-27 (translated to 18563), Spark determines the stride size as follows:

 

(18563L / 1000L) - (-15611 / 1000L) = 33

Starting from the lower bound, doing strides of 33, you'll end up at 2017-07-08. This is over 3 years of extra data that will go into the last partition, and depending on the shape of the data could cause a very long running task at the end of a job.

 

Using the formula I'm proposing, you'd get:

((18563L / 1000F) - (-15611 / 1000F)).toLong = 34

This would put the upper bound at 2020-04-02, which is much closer to the original supplied upper bound. This is the best you can do to get as close as possible to the upper bound (without adjusting the number of partitions). For example, a stride size of 35 would go well past the supplied upper bound (over 2 years, 2022-11-22).

 

In the above example, there is only a difference of 1 between the stride size using the current formula and the stride size using the proposed formula, but with greater distance between the lower and upper bounds, or a lower number of partitions, the difference can be much greater.
Bug: Preserve broadcast nested loop join output partitioning and ordering
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 02/Mar/21 09:42
Affects Version/s: 3.2.0

`BroadcastNestedLoopJoinExec` does not preserve `outputPartitioning` and `outputOrdering` right now. But it can preserve the streamed side partitioning and ordering when possible. This can help avoid shuffle and sort in later stage, if there's join and aggregation in the query.
Bug: Support nested column in Spark ORC vectorized readers
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 25/Mar/21 08:55
Affects Version/s: 3.2.0

The task is to support nested column type in Spark ORC vectorized reader. Currently ORC vectorized reader does not support nested column type (struct, array and map). We implemented nested column vectorized reader for FB-ORC in our internal fork of Spark. We are seeing performance improvement compared to non-vectorized reader when reading nested columns. In addition, this can also help improve the non-nested column performance when reading non-nested and nested columns together in one query.

 

ORC:

[https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala#L138]
Bug: Add nested column scan in DataSourceReadBenchmark
Status: Open | Priority: Minor | Resolution: 
Created: 31/Mar/21 20:33
Affects Version/s: 3.2.0

For the table scan benchmark `DataSourceReadBenchmark.scala`, we are only testing atomic data type but not nested column data type (array/struct/map) - [https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala#L544-L567] . We should add scan benchmark for nested column data type as well.
Bug: Old import of LZ4 package inside CompressionCodec.scala 
Status: Resolved | Priority: Minor | Resolution: Not A Problem
Created: 30/Mar/21 08:06
Affects Version/s: 2.4.0

This commit should upgrade the version of the LZ4 package: [https://github.com/apache/spark/commit/b78cf13bf05f0eadd7ae97df84b6e1505dc5ff9f] 

The dependency was changed. However, inside the file [CompressionCodec.scala |https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/io/CompressionCodec.scala] the old import referencing net.jpountz.lz4 (where versions up to 1.3. are held) remains. 

Because of probably backward compatibility, the newer version of org.lz4 package still contains net.jpountz.lz4 ([https://github.com/lz4/lz4-java/tree/master/src/java/net/jpountz/lz4)]. Therefore this import does not cause problems at a first sight.
Bug: Executor log pagination UI blocked by privacy blockers
Status: Open | Priority: Minor | Resolution: 
Created: 30/Mar/21 00:12
Affects Version/s: 3.0.0

Executor log pagination is broken when using privacy blockers, such as uBlock Origin. For example, [EasyList|https://easylist.to/] has an [EasyPrivacy|https://easylist.to/easylist/easyprivacy.txt] list that blocks {{/log-view.}} Unfortunately, we use {{/static/log-view.js}} to load logs: https://github.com/apache/spark/blob/c91a75620465c5606b837a595d6cc55c7219026f/core/src/main/scala/org/apache/spark/ui/UIUtils.scala#L240
Bug: Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with describe pods output
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 25/Mar/21 13:17
Affects Version/s: 3.2.0

As came up in https://github.com/apache/spark/pull/31923#issuecomment-806706204 we can extend  "EXTRA LOGS FOR THE FAILED TEST" section with describe pods output.
Bug: Sort output rows in SQLQueryTestSuite even if a plan has a Sort node
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 24/Mar/21 01:09
Affects Version/s: 3.0.3

This ticket aims at sorting output rows in SQLQueryTestSuite even if a given plan has a `Sort` node. In that case, the current logic skips sorting output rows (see the code below for details):
https://github.com/apache/spark/blob/0494dc90af48ce7da0625485a4dc6917a244d580/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala#L503-L522

But, I think we still need to sort output rows if the node just sort output rows by partial columns (instead of all columns). The current logic can lead to a flaky test behaviour.
Bug: Close Hive session state should use withHiveState
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Mar/21 07:06
Affects Version/s: 3.2.0

Some reason:

1. Shutdown hook is invoked using different thread.
 2. Hive may use metasotre client again during closing

Otherwise, we may get such expcetion with custom hive metastore version
{code:java}
21/03/24 13:26:18 INFO session.SessionState: Failed to remove classloaders from DataNucleus
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101)
	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3367)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3406)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3386)
	at org.apache.hadoop.hive.ql.session.SessionState.unCacheDataNucleusClassLoaders(SessionState.java:1546)
	at org.apache.hadoop.hive.ql.session.SessionState.close(SessionState.java:1536)
	at org.apache.spark.sql.hive.client.HiveClientImpl.closeState(HiveClientImpl.scala:172)
	at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$new$1(HiveClientImpl.scala:175)
	at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:214)
	at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$2(ShutdownHookManager.scala:188)
{code}
Bug: JDBCRelation columnPartition function includes the first stride in the lower partition
Status: Resolved | Priority: Minor | Resolution: Not A Problem
Created: 23/Mar/21 20:52
Affects Version/s: 3.0.0

Currently, columnPartition in JDBCRelation contains logic that adds the first stride into the lower partition. Because of this, the lower bound isn't used as the ceiling for the lower partition.

For example, say we have data 0-10, 10 partitions, and the lowerBound is set to 1. The lower/first partition should contain anything < 1. However, in the current implementation, it would include anything < 2.

A possible easy fix would be changing the following code on line 132:

currentValue += stride

To:

if (i != 0) currentValue += stride

Or include currentValue += stride within the if statement on line 131... although this creates a pretty bad looking side-effect.
Bug: run JavaSQLDataSourceExample failed with Exception in runBasicDataSourceExample().
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Mar/21 09:50
Affects Version/s: 3.0.1

run JavaSparkSQLExample failed with Exception in runBasicDataSourceExample().

when excecute 'peopleDF.write().partitionBy("favorite_color").bucketBy(42,"name").saveAsTable("people_partitioned_bucketed");'

throws Exception: 'Exception in thread "main" org.apache.spark.sql.AnalysisException: partition column favorite_color is not defined in table people_partitioned_bucketed, defined table columns are: age, name;'
Bug: run JavaSparkSQLExample failed with Exception.
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 16/Mar/21 09:39
Affects Version/s: 3.0.1

run JavaSparkSQLExample failed with Exception.

The Exception is thrown  in function runDatasetCreationExample, when execute ‘spark.read().json(path).as(personEncoder)’.

The exception is  'Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot up cast `age` from bigint to int.'
Bug: consider markdownlint
Status: Open | Priority: Minor | Resolution: 
Created: 23/Mar/21 15:11
Affects Version/s: 3.1.1

There's apparently a tool called markdownlint which seems to be integrated w/ Visual Studio Code. It suggests, among other things, to use {{*}} instead of {{-}} for top level bullets:

https://github.com/DavidAnson/markdownlint/blob/v0.23.1/doc/Rules.md#md004 
https://github.com/apache/spark/pull/30679#issuecomment-804971370
Bug: Code-gen broadcast nested loop join (left outer/right outer)
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 11/Mar/21 02:35
Affects Version/s: 3.2.0

We saw 1x run-time improvement for code-gen broadcast nested loop inner join (https://issues.apache.org/jira/browse/SPARK-34620 ). Similarly let's add code-gen for left outer (build right side), and right outer (build left side) as well here.
Bug: registerFunction shouldnt be logging warning message for same function being re-registered
Status: In Progress | Priority: Minor | Resolution: 
Created: 19/Mar/21 16:12
Affects Version/s: 3.1.1

{code:java}
test("function registry warning") {
 implicit val ss = spark
 import ss.implicits._
 val dd = Seq((1),(2)).toDF("a")
 val dd1 = udf((i: Int) => i*2)
 (1 to 4).foreach {
   _ =>
     dd.sparkSession.udf.register("function", dd1)
     Thread.sleep(1000)
 }
 dd.withColumn("aa", expr("function(a)")).show(10)
}{code}
logs:

21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; some spark core configurations may not take effect.

21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; the static sql configurations will not take effect.
 21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; some spark core configurations may not take effect.
 21/03/19 22:39:43 WARN SimpleFunctionRegistry : The function function replaced a previously registered function.
 21/03/19 22:39:44 WARN SimpleFunctionRegistry : The function function replaced a previously registered function.
 21/03/19 22:39:45 WARN SimpleFunctionRegistry : The function function replaced a previously registered function.
 +----+--+
|a|aa|

+----+--+
|1|2|
|2|4|

+----+--+


Basically in the FunctionRegistry implementation
{code:java}
override def registerFunction(
 name: FunctionIdentifier,
 info: ExpressionInfo,
 builder: FunctionBuilder): Unit = synchronized {
 val normalizedName = normalizeFuncName(name)
 val newFunction = (info, builder)
 functionBuilders.put(normalizedName, newFunction) match {
 case Some(previousFunction) if previousFunction != newFunction =>
 logWarning(s"The function $normalizedName replaced a previously registered function.")
 case _ =>
 }
}{code}
The *previousFunction != newFunction* equality comparison is incorrect
Bug: Code-gen broadcast nested loop join (left semi/left anti)
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 11/Mar/21 02:36
Affects Version/s: 3.2.0

We saw 1x run-time improvement for code-gen broadcast nested loop inner join (https://issues.apache.org/jira/browse/SPARK-34620 ). Similarly let's add code-gen for left semi and left anti join here (build right side).
Bug: RebaseDateTime loadRebaseRecords should use Spark classloader instead of context
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 17/Mar/21 07:07
Affects Version/s: 3.1.1

With custom `spark.sql.hive.metastore.version` and `spark.sql.hive.metastore.jars`.

Spark would use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, if `RebaseDateTime` is initialized the first time then context class loader is `IsolatedClientLoader`. Such error msg would throw:

{code:java}
java.lang.IllegalArgumentException: argument "src" is null
  at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413)
  at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157)
  at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187)
  at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186)
  at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267)
  at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269)
  at org.apache.spark.sql.catalyst.util.RebaseDateTime$.<init>(RebaseDateTime.scala:291)
  at org.apache.spark.sql.catalyst.util.RebaseDateTime$.<clinit>(RebaseDateTime.scala)
  at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109)
  at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95)
  at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94)
  at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138)
  at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661)
  at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785)
  at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826)
{code}
Bug: The `change-scala- version.sh` script not replaced scala.version property correctly
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 17/Mar/21 07:22
Affects Version/s: 3.2.0

Execute the following commands in order
 # dev/change-scala-version.sh 2.13

 # dev/change-scala-version.sh 2.12

 # git status

there will generate git diff as follow:
{code:java}
diff --git a/pom.xml b/pom.xml
index ddc4ce2f68..f43d8c8f78 100644
--- a/pom.xml
+++ b/pom.xml
@@ -162,7 +162,7 @@
     <commons.math3.version>3.4.1</commons.math3.version>
     <!-- managed up from 3.2.1 for SPARK-11652 -->
     <commons.collections.version>3.2.2</commons.collections.version>
-    <scala.version>2.12.10</scala.version>
+    <scala.version>2.13.5</scala.version>
     <scala.binary.version>2.12</scala.binary.version>
     <scalatest-maven-plugin.version>2.0.0</scalatest-maven-plugin.version>
     <scalafmt.parameters>--test</scalafmt.parameters>
{code}
seem 'scala.version' property was not replaced correctly
Bug: Do not capture maven config for views
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 17/Mar/21 02:07
Affects Version/s: 3.2.0

Due to the bad network, we always use the thirdparty maven repo to run test. e.g.,
{code:java}
build/sbt "test:testOnly *SQLQueryTestSuite" -Dspark.sql.maven.additionalRemoteRepositories=xxxxx
{code}
 
It's failed with such error msg
```
[info] - show-tblproperties.sql *** FAILED *** (128 milliseconds)
[info] show-tblproperties.sql
[info] Expected "...rredTempViewNames [][]", but got "...rredTempViewNames [][
[info] view.sqlConfig.spark.sql.maven.additionalRemoteRepositories xxxxx]" Result did not match for query #6
[info] SHOW TBLPROPERTIES view (SQLQueryTestSuite.scala:464)
```

It's not necessary to capture the maven config to view since it's a session level config.
Bug: change-scala-version.sh fails to modify pom.xml if it changes the version from 2.13 to 2.12
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 17/Mar/21 07:20
Affects Version/s: 3.2.0

If we change the Scala version to 2.13 and then change the version to 2.12 using change-scala-version, it fails to modify pom.xml and fails to build.

{code}
$ dev/change-scala-version.sh 2.13
$ dev/change-scala-version.sh 2.12
$ git diff pom.xml
diff --git a/pom.xml b/pom.xml
index 2efa3491a6..79994d0c44 100644
--- a/pom.xml
+++ b/pom.xml
@@ -162,7 +162,7 @@
     <commons.math3.version>3.4.1</commons.math3.version>
     <!-- managed up from 3.2.1 for SPARK-11652 -->
     <commons.collections.version>3.2.2</commons.collections.version>
-    <scala.version>2.12.10</scala.version>
+    <scala.version>2.13.5</scala.version>
     <scala.binary.version>2.12</scala.binary.version>
     <scalatest-maven-plugin.version>2.0.0</scalatest-maven-plugin.version>
     <scalafmt.parameters>--test</scalafmt.parameters>
{code}
Bug: Faster execution for broadcast nested loop join (left semi/anti with no condition)
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 13/Mar/21 01:31
Affects Version/s: 3.2.0

For `BroadcastNestedLoopJoinExec` left semi and left anti join without condition. If we broadcast left side. Currently we check whether every row from broadcast side has a match or not by iterating broadcast side a lot of time - [https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastNestedLoopJoinExec.scala#L256-L275] . This is unnecessary, as there's no condition, and we only need to check whether stream side is empty or not. Create this Jira to add the optimization. This can boost the affected query execution performance a lot.
Bug: Clean up deprecated API usage related to JUnit
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 12/Mar/21 03:24
Affects Version/s: 3.2.0

Maven compile compilation warnings as follows:

 
{code:java}
[WARNING] [Warn] /spark/common/network-common/src/test/java/org/apache/spark/network/client/TransportClientFactorySuite.java:231: org.junit.rules.ExpectedException中的none()已过时
[WARNING] [Warn] /spark/common/network-common/src/test/java/org/apache/spark/network/client/TransportClientFactorySuite.java:247: org.junit.rules.ExpectedException中的none()已过时
[WARNING] [Warn] /spark/common/network-common/src/test/java/org/apache/spark/network/crypto/TransportCipherSuite.java:84: org.junit.Assert中的<T>assertThat(T,org.hamcrest.Matcher<? super T>)已过时
[WARNING] [Warn] /spark/launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java:43: org.junit.rules.ExpectedException中的none()已过时
[WARNING] [Warn] /spark/core/src/test/java/org/apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java:542: org.junit.Assert中的<T>assertThat(T,org.hamcrest.Matcher<? super T>)已过时
[WARNING] [Warn] /spark/core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java:227: org.junit.Assert中的<T>assertThat(T,org.hamcrest.Matcher<? super T>)已过时
[WARNING] [Warn] /spark/core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java:235: org.junit.Assert中的<T>assertThat(T,org.hamcrest.Matcher<? super T>)已过时
[WARNING] [Warn] /spark/core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java:254: org.junit.Assert中的<T>assertThat(T,org.hamcrest.Matcher<? super T>)已过时
[WARNING] [Warn] /spark/sql/core/src/test/java/test/org/apache/spark/sql/JavaDatasetSuite.java:915: org.junit.rules.ExpectedException中的none()已过时

{code}
 

"已过时" means  Deprecated
Bug: Do not show Statistics(sizeInBytes=8.0 EiB) if we don't have valid stats
Status: Resolved | Priority: Minor | Resolution: Later
Created: 12/Mar/21 08:02
Affects Version/s: 3.2.0

In cast explian mode, we might get such explain string if the table is external and we have not stats.

 
{code:java}
== Optimized Logical Plan == 
GlobalLimit 21, Statistics(sizeInBytes=1008.0 B, rowCount=21)
 +- LocalLimit 21, Statistics(sizeInBytes=12.0 EiB)
 +- Project [cast(c1#52 as string) AS c1#259, pt#53], Statistics(sizeInBytes=12.0 EiB)
 +- Relation default.pt1[c1#52,pt#53] parquet, Statistics(sizeInBytes=8.0 EiB)
{code}
 

The reason is we use the `Long.MaxValue` as the stats with conf `spark.sql.defaultSizeInBytes`. It would be better to hide the default stat.
Bug: The logForFailedTest throws an exception when driver is not started
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 13/Mar/21 14:37
Affects Version/s: 3.2.0

In SPARK-34426 the logForFailedTest method is introduced to add the driver and executors log to the integration-tests.log but when the driver failed to start an exception will be thrown as the list of pods is empty but we still try to access the first item from the list:


{noformat}
- PVs with local storage *** FAILED ***
  java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
  at java.util.ArrayList.rangeCheck(ArrayList.java:659)
  at java.util.ArrayList.get(ArrayList.java:435)
  at org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.logForFailedTest(KubernetesSuite.scala:83)
  at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:181)
  at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:188)
  at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:200)
  at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
  at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:200)
  at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:182)
  at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:61)
 {noformat}
Bug: Code-gen broadcast nested loop join (defaultJoin)
Status: Open | Priority: Minor | Resolution: 
Created: 11/Mar/21 02:38
Affects Version/s: 3.2.0

For full outer join, and all other join types cases, e.g. left outer + build left, right outer + build right, etc. Spark is using `BroadcastNestedLoopJoinExec.defaultJoin()` to execute those. Create this Jira to add code-gen for it.
Bug: Code-gen broadcast nested loop join (inner/cross)
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 04/Mar/21 12:22
Affects Version/s: 3.2.0

`BroadcastNestedLoopJoinExec` does not have code-gen, and we can potentially boost the CPU performance for this operator if we add code-gen for it. [https://databricks.com/blog/2017/02/16/processing-trillion-rows-per-second-single-machine-can-nested-loop-joins-fast.html] also showed the evidence in one fork.