av9ash/GitBugs huggingface dataset:
Bug: LikeSimplification should be skipped if pattern contains any escapeChar
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Dec/20 12:13
Affects Version/s: 3.1.0

LikeSimplification rule does not work correctly for many cases that have patterns containing escape characters:
{code:sql}
SELECT s LIKE 'm%aca' ESCAPE '%' from t;
SELECT s LIKE 'maacaa' ESCAPE 'a' FROM t;
{code}
Bug: ArrayData specialization to enable RowEncoder primitive arrays support
Status: In Progress | Priority: Major | Resolution: 
Created: 07/Dec/20 11:42
Affects Version/s: 3.0.1

Currently, even non-nullable array fields are represented as WrappedArray.ofRef in external rows. It leads to memory footprint, as well as unnecessary cycles in boxing/unboxing operations. Ideally, if arrays are non-nullable and contain primitive values, they should be represented as specialized less-overhead versions of WrappedArray.
Bug: Require exact matched partition spec to schema in ADD/DROP PARTITION
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Dec/20 09:13
Affects Version/s: 3.1.0

The V1 implementation of ALTER TABLE .. ADD/DROP PARTITION fails when the partition spec doesn't exactly match to the partition schema:
{code:sql}
ALTER TABLE tab1 ADD PARTITION (A='9')
Partition spec is invalid. The spec (a) must match the partition spec (a, b) defined in table '`dbx`.`tab1`';
org.apache.spark.sql.AnalysisException: Partition spec is invalid. The spec (a) must match the partition spec (a, b) defined in table '`dbx`.`tab1`';
	at org.apache.spark.sql.catalyst.catalog.SessionCatalog.$anonfun$requireExactMatchedPartitionSpec$1(SessionCatalog.scala:1173)
	at org.apache.spark.sql.catalyst.catalog.SessionCatalog.$anonfun$requireExactMatchedPartitionSpec$1$adapted(SessionCatalog.scala:1171)
	at scala.collection.immutable.List.foreach(List.scala:392)
{code}
for a table partitioned by "a", "b" but the V2 implementation add the wrong partition silently.
Bug: Show Slowpoke notifications in SBT tests
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Dec/20 04:23
Affects Version/s: 3.0.1

When the tests/code has bug and enters the infinite loop, it is hard to tell which test cases hit some issues from the log, especially when we are running the tests in parallel. It would be nice to show the Slowpoke notifications.
Bug: Check SQLContext.tables() for V2 session catalog
Status: Open | Priority: Major | Resolution: 
Created: 05/Dec/20 17:02
Affects Version/s: 3.1.0

V1 ShowTablesCommand is hard coded in SQLContext:
https://github.com/apache/spark/blob/a088a801ed8c17171545c196a3f26ce415de0cd1/sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala#L671
The ticket aims to checks tables() behavior for V2 session catalog.
Bug: Enable ShuffleBlockPusher to stop pushing blocks for a particular shuffle partition
Status: Open | Priority: Major | Resolution: 
Created: 04/Dec/20 20:43
Affects Version/s: 3.1.0

{{ShuffleBlockPusher}}, which was introduced in SPARK-32917 stops pushing shuffle blocks for the entire shuffle when it receives "TOO Late" exception. However, with the change [https://github.com/apache/spark/pull/30433], there is also a need to stop pushing shuffle blocks for a particular reduce partition. Refer https://github.com/apache/spark/pull/30433#discussion_r533694433
Bug: Improve the doc of spark.sql.ansi.enabled
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 03/Dec/20 15:28
Affects Version/s: 3.1.0

As there are more and more new features under the SQL configuration spark.sql.ansi.enabled, we should make it more clear about:
1. what exactly it is
2. where user can find all the features of the ANSI mode
3. whether all the feature exactly from the SQL standard
Bug: Misleading error from ALTER TABLE .. PARTITION for non-supported partition management table
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 03/Dec/20 17:02
Affects Version/s: 3.1.0

For a V2 table that doesn't support partition management, ALTER TABLE .. ADD/DROP PARTITION throws misleading exception:
{code:java}
PartitionSpecs are not resolved;;
'AlterTableAddPartition [UnresolvedPartitionSpec(Map(id -> 1),None)], false
+- ResolvedTable org.apache.spark.sql.connector.InMemoryTableCatalog@2fd64b11, ns1.ns2.tbl, org.apache.spark.sql.connector.InMemoryTable@5d3ff859

org.apache.spark.sql.AnalysisException: PartitionSpecs are not resolved;;
'AlterTableAddPartition [UnresolvedPartitionSpec(Map(id -> 1),None)], false
+- ResolvedTable org.apache.spark.sql.connector.InMemoryTableCatalog@2fd64b11, ns1.ns2.tbl, org.apache.spark.sql.connector.InMemoryTable@5d3ff859

	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis(CheckAnalysis.scala:50)
	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis$(CheckAnalysis.scala:49)
{code}

The error should say that the table doesn't support partition management.
Bug: Add canDeleteWhere to SupportsDelete
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Dec/20 11:51
Affects Version/s: 3.1.0

The only way to support delete statements right now is to implement \{{SupportsDelete}}. According to its Javadoc, that interface is meant for cases when we can delete data without much effort (e.g. like deleting a complete partition in a Hive table). It is clear we need a more sophisticated API for row-level deletes. That's why it would be beneficial to add a method to \{{SupportsDelete}} so that Spark can check if a source can easily delete data with just having filters or it will need a full rewrite later on. This way, we have more control in the future.
Bug: Add new function DATE_FROM_UNIX_DATE and UNIX_DATE
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 03/Dec/20 09:36
Affects Version/s: 3.1.0

h2. What changes were proposed in this pull request?

Add new functions DATE_FROM_UNIX_DATE and UNIX_DATE for conversion between Date type and Numeric types.


h2. Why are the changes needed?

1. Explicit conversion between Date type and Numeric types is disallowed in ANSI mode. We need to provide new functions for users to complete the conversion.

2. We have introduced new functions from Bigquery for conversion between Timestamp type and Numeric types: TIMESTAMP_SECONDS, TIMESTAMP_MILLIS, TIMESTAMP_MICROS , UNIX_SECONDS, UNIX_MILLIS, and UNIX_MICROS. It makes sense to add functions for conversion between Date type and Numeric types as well.
Bug: Add new function UNIX_SECONDS, UNIX_MILLIS and UNIX_MICROS
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Dec/20 08:01
Affects Version/s: 3.1.0

As https://github.com/apache/spark/pull/28534 adds functions from BigQuery for converting numbers to timestamp, we should add functions UNIX_SECONDS, UNIX_MILLIS and UNIX_MICROS for converting timestamp to numbers.
Bug: Subexpression elimination for whole-stage codegen in Filter
Status: In Progress | Priority: Major | Resolution: 
Created: 02/Dec/20 06:37
Affects Version/s: 3.1.0

We made subexpression elimination available for whole-stage codegen in ProjectExec. Another one operator that frequently runs into subexpressions, is Filter. We should also make whole-stage codegen subexpression elimination in FilterExec too.
Bug: InsertIntoHiveTable generate HDFS file with invalid user
Status: Open | Priority: Minor | Resolution: 
Created: 17/Dec/20 09:52
Affects Version/s: 2.3.2

*Arch:* Hive on Spark.

 

*Version:* Spark 2.3.2

 

*Conf:*

Enable user impersonation

hive.server2.enable.doAs=true

 

*Scenario:*

Thriftserver is running with loginUser A, and Task  run as User A too.

Client execute SQL with user B

 

Data generated by sql "insert into TABLE  \[tbl\] select XXX form XXXX." is written to HDFS on executor, executor doesn't know B.

 

*{color:#de350b}So the user file written to HDFS will be user A which should be B.{color}*

 

I also check the inplementation of Spark 3.0.0, It could have the same issue.
Bug: Eliminate deprecated usage since Scala 2.13
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Dec/20 08:11
Affects Version/s: 3.2.0

SPARK-33775 Suppress compilation warnings about method, value, type, object, trait, inheritance class deprecated usage since Scala 2.13 in SparkBuild.scala

 

We should fix them step by step, and then remove the suppression rules.
Bug: grouping__id() result does not consistent with hive's version < 2.3
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 15/Dec/20 12:42
Affects Version/s: 2.4.3

See this [https://cwiki.apache.org/confluence/display/Hive/Enhanced+Aggregation%2C+Cube%2C+Grouping+and+Rollup]

Hive's grouping__id method made a change since hive version 2.3.0. Now spark does not declare this inconsistency with Hive, which may make user believe they're safe from migrating their query from Hive 1.x to Spark, but which is wrong.

I guess we should note this difference in Hive migration guide, and add a configuration to let grouping__id to use hive 1.x compatible algorithm
Bug: Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 09/Dec/20 09:05
Affects Version/s: 3.1.0

Add make_date/make_timestamp/make_interval into the doc of ANSI Compliance. So that users can know that these function throw runtime exception if the result is not valid under ANSI mode.
Bug: Error display of Spark SQL
Status: Resolved | Priority: Minor | Resolution: Incomplete
Created: 08/Dec/20 09:02
Affects Version/s: 2.3.2

SQL displayed in the following table shows it starts from a wrong time.

I can't find any information from thriftserver log about this SQL.

 


|ID|Description|submited|Duration| | |Failed Job ID|
|1243324|[1243324|https://76.65.40.144:31943/bigdata/Yarn/ResourceManager/36/proxy/application_1606813567908_0161/SQL/execution/?id=1243324]|1970/01/01 07:59:59|446504.9 h| | |[[298314]|https://76.65.40.144:31943/bigdata/Yarn/ResourceManager/36/proxy/application_1606813567908_0161/jobs/job?id=298314]|
Bug: Result of add "unused-locals" compile arg to scalac
Status: Open | Priority: Minor | Resolution: 
Created: 08/Dec/20 05:45
Affects Version/s: 3.1.0

Try to add "unused-locals" compile arg to scalac and collect the warning informations, there are total 184 files and 296 compile warnings about  "unused-locals":
 * core: 92 warnings
 * mllib-local\mllib:  42 warnings
 * graphx: 6 warnings
 * streaming: 14 warnings
 * Sql\hive\avro: 81 warnings
 * Yarn\mesos\k8s: 23 warnings
 * kafka: 3 warnings
 * kinesis-asl: 4 warnings
 * examples: 31  warnings
Bug: Add Scala, Maven and Zinc caches to GitHub Action job to publish snapshot
Status: Resolved | Priority: Minor | Resolution: Later
Created: 07/Dec/20 01:43
Affects Version/s: 2.4.7

We should also better add caches of Scala, Maven and Zinc, e.g.)

{code}
    - name: Cache Scala, Maven and Zinc
      uses: actions/cache@v2
      with:
        path: |
          build/apache-maven-*
          build/zinc-*
          build/scala-*
        key: build-${{ hashFiles('**/pom.xml', 'build/mvn') }}
        restore-keys: |
          build-
{code}
Bug: Fix misleading message for uncaching when createOrReplaceTempView is called
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 04/Dec/20 18:40
Affects Version/s: 3.1.0

To repro:

{code:java}
scala> sql("CREATE TABLE table USING parquet AS SELECT 2")
res0: org.apache.spark.sql.DataFrame = []                                       

scala> val df = spark.table("table")
df: org.apache.spark.sql.DataFrame = [2: int]

scala> df.createOrReplaceTempView("t2")
20/12/04 10:16:24 WARN CommandUtils: Exception when attempting to uncache $name
org.apache.spark.sql.AnalysisException: Table or view not found: t2;;
'UnresolvedRelation [t2], [], false

	at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$1(CheckAnalysis.scala:113)
	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$1$adapted(CheckAnalysis.scala:93)
	at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:183)
	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.checkAnalysis(CheckAnalysis.scala:93)
	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.checkAnalysis$(CheckAnalysis.scala:90)
	at org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:152)
	at org.apache.spark.sql.catalyst.analysis.Analyzer.$anonfun$executeAndCheck$1(Analyzer.scala:172)
	at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.markInAnalyzer(AnalysisHelper.scala:214)
	at org.apache.spark.sql.catalyst.analysis.Analyzer.executeAndCheck(Analyzer.scala:169)
	at org.apache.spark.sql.execution.QueryExecution.$anonfun$analyzed$1(QueryExecution.scala:73)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
	at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:138)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:768)
	at org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:138)
	at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:73)
	at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:71)
	at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:63)
	at org.apache.spark.sql.Dataset$.$anonfun$ofRows$1(Dataset.scala:90)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:768)
	at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:88)
	at org.apache.spark.sql.DataFrameReader.table(DataFrameReader.scala:889)
	at org.apache.spark.sql.SparkSession.table(SparkSession.scala:589)
	at org.apache.spark.sql.internal.CatalogImpl.uncacheTable(CatalogImpl.scala:476)
	at org.apache.spark.sql.execution.command.CommandUtils$.uncacheTableOrView(CommandUtils.scala:392)
	at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:124)

{code}

It shouldn't log because `t2` does not exist yet.
Bug: Numerical product aggregation
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 06/Dec/20 12:40
Affects Version/s: 2.4.7

There is currently no facility in {{spark.sql.functions}} to allow computation of the product of all numbers in a grouping expression. Such a facility would likely be useful when computing statistical quantities such as the combined probability of a set of independent events, or in financial applications when calculating a cumulative interest rate.

Although it is certainly possible to emulate this by an expression of the form {{exp(sum(log(column)))}}, this has a number of significant drawbacks:
 * It involves computationally costly functions (exp, log)
 * It is more verbose than something like {{product(column)}}
 * It is more prone to numerical inaccuracies when handling quantities that are close to one than by directly multiplying a set of numbers
 * It will not handle zeros or negative numbers cleanly

I am currently developing an addition to {{sql.functions}}, which involves [a new Catalyst aggregation expression|https://github.com/rwpenney/spark/blob/feature/agg-product/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Product.scala]. This needs some additional testing, and I hope to issue a pull-request soon.
Bug: Fix error test SparkLauncherSuite.testSparkLauncherGetError
Status: Resolved | Priority: Minor | Resolution: Invalid
Created: 28/Dec/20 02:18
Affects Version/s: 3.0.1

org.apache.spark.launcher.SparkLauncherSuite.testSparkLauncherGetError get failed everytime when executing, note that it's not a flaky test because it failed everytime.

```

java.lang.AssertionErrorjava.lang.AssertionError at org.junit.Assert.fail(Assert.java:87) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertTrue(Assert.java:53) at org.apache.spark.launcher.SparkLauncherSuite.testSparkLauncherGetError

```
Bug: Python-friendly dtypes for pyspark dataframes
Status: Open | Priority: Minor | Resolution: 
Created: 31/Dec/20 10:45
Affects Version/s: 3.2.0

The pyspark.sql.DataFrame.dtypes attribute contains string representations of the column datatypes in terms of JVM datatypes. However, for a python user it is a significant mental step to translate these to the corresponding python types encountered in UDF's and collected dataframes. This holds in particular for nested composite datatypes (array, map and struct). It is proposed to provide python-friendly dtypes in pyspark (as an addition, not a replacement) in which array<>, map<> and struct<> are translated to [], {} and Row().

Sample code, including tests, is available as [gist on github|https://gist.github.com/vtslab/81ded1a7af006100e00bf2a4a70a8147]. More explanation is provided at: [https://yaaics.blogspot.com/2020/12/python-friendly-dtypes-for-pyspark.html]

If this proposal finds sufficient support, I can provide a PR.
Bug: Cannot access to column after left semi join  and left join
Status: Resolved | Priority: Minor | Resolution: Not A Problem
Created: 21/Dec/20 21:09
Affects Version/s: 3.0.0

Cannot access to column after left semi join and left join

{code}
val col = "c1"
val df = Seq((1, "a"),(2, "a"),(3, "a"),(4, "a")).toDF(col, "c2")
val df2 = Seq(1).toDF(col)

val semiJoin = df.join(df2, df(col) === df2(col), "left_semi")

val left = df.join(semiJoin, df(col) === semiJoin(col), "left")

left.show

+---+---+----+----+
| c1| c2|  c1|  c2|
+---+---+----+----+
|  1|  a|   1|   a|
|  2|  a|null|null|
|  3|  a|null|null|
|  4|  a|null|null|
+---+---+----+----+

left.select(semiJoin(col))

+---+
| c1|
+---+
|  1|
|  2|
|  3|
|  4|
+---+

left.select(df(col))

+---+
| c1|
+---+
|  1|
|  2|
|  3|
|  4|
+---+
{code}
Bug: Can't set option 'cross' in join method.
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Dec/20 05:27
Affects Version/s: 3.0.1

[The PySpark documentation|https://spark.apache.org/docs/3.0.1/api/python/pyspark.sql.html#pyspark.sql.DataFrame.join] says "Must be one of: inner, cross, outer, full, fullouter, full_outer, left, leftouter, left_outer, right, rightouter, right_outer, semi, leftsemi, left_semi, anti, leftanti and left_anti."
However, I get the following error when I set the cross option.

 
{code:java}
scala> val df1 = spark.createDataFrame(Seq((1,"a"),(2,"b")))
df1: org.apache.spark.sql.DataFrame = [_1: int, _2: string]
scala> val df2 = spark.createDataFrame(Seq((1,"A"),(2,"B"), (3, "C")))
df2: org.apache.spark.sql.DataFrame = [_1: int, _2: string]
scala> df1.join(right = df2, usingColumns = Seq("_1"), joinType = "cross").show()
java.lang.IllegalArgumentException: requirement failed: Unsupported using join type Cross
 at scala.Predef$.require(Predef.scala:281)
 at org.apache.spark.sql.catalyst.plans.UsingJoin.<init>(joinTypes.scala:106)
 at org.apache.spark.sql.Dataset.join(Dataset.scala:1025)
 ... 53 elided

{code}
Bug: Cleanup "Unicode escapes in triple quoted strings are deprecated" compilation warnings
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Dec/20 07:08
Affects Version/s: 3.2.0

There are total 15 compilation warnings about this
{code:java}
[WARNING] /spark-source/core/src/main/scala/org/apache/spark/util/Utils.scala:2930: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/core/src/main/scala/org/apache/spark/util/Utils.scala:2931: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/core/src/main/scala/org/apache/spark/util/Utils.scala:2932: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/core/src/main/scala/org/apache/spark/util/Utils.scala:2933: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/core/src/main/scala/org/apache/spark/util/Utils.scala:2934: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/core/src/main/scala/org/apache/spark/util/Utils.scala:2935: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/core/src/main/scala/org/apache/spark/util/Utils.scala:2936: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/core/src/main/scala/org/apache/spark/util/Utils.scala:2937: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVExprUtils.scala:82: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/csv/CSVExprUtilsSuite.scala:32: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/csv/CSVExprUtilsSuite.scala:79: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ParserUtilsSuite.scala:97: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ParserUtilsSuite.scala:101: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonParsingOptionsSuite.scala:76: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
[WARNING] /spark-source/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonParsingOptionsSuite.scala:83: Unicode escapes in triple quoted strings are deprecated, use the literal character instead
{code}
Bug: JDBC SQL TIME type represents incorrectly as TimestampType, it should be physical Int in millis
Status: Resolved | Priority: Minor | Resolution: Invalid
Created: 23/Dec/20 06:53
Affects Version/s: 2.4.3

Currently, for JDBC, SQL TIME type represents incorrectly as Spark TimestampType. This should be represent as physical int in millis Represents a time of day, with no reference to a particular calendar, time zone or date, with a precision of one millisecond. It stores the number of milliseconds after midnight, 00:00:00.000.

We encountered the issue of Avro logical type of `TimeMillis` not being converted correctly to Spark `Timestamp` struct type using the `SchemaConverters`, but it converts to regular `int` instead. Reproducible by ingest data from MySQL table with a column of TIME type: Spark JDBC dataframe will get the correct type (Timestamp), but enforcing our avro schema (`{"type": "int"," logicalType": "time-millis"}`) externally will fail to apply with the following exception:

{{java.lang.RuntimeException: java.sql.Timestamp is not a valid external type for schema of int}}
Bug: Include Comments for StructType toDDL function
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 19/Dec/20 07:32
Affects Version/s: 3.0.1

{{StructField.toDDL returns a DDL string including its comment. However, when there is a nested StructType toDDL does not include nested fields' comment.toDDL function should include nested fields' comment.}}