av9ash/GitBugs huggingface dataset:
Bug: Alter view does not preserve column case
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/Oct/21 17:22
Affects Version/s: 3.2.0

On running an `alter view` command, the column case is not preserved.

Repro:

 
{code:java}
scala> sql("create view v as select 1 as A, 1 as B")
res2: org.apache.spark.sql.DataFrame = []

scala> sql("describe v").collect.foreach(println)
[A,int,null]
[B,int,null]

scala> sql("alter view v as select 1 as C, 1 as D")
res4: org.apache.spark.sql.DataFrame = []

scala> sql("describe v").collect.foreach(println)
[c,int,null]
[d,int,null]
 
{code}
Bug: Modify the assignment logic of dirFetchRequests variables
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 17/Oct/21 09:45
Affects Version/s: 3.1.2

In the ShuffleBlockFetcherIterator.fetchHostLocalBlocks method, we generate dirFetchRequests based on externalShuffleServiceEnabled. But in fact, the MapStatus object generated in the shuffle write phase had already generated the BlockManagerId object according to externalShuffleServiceEnabled in the BlockManager.initialize method.

So we don't need to judge it again.
Bug: Reduce duplication in TaskMemoryManager.acquireExecutionMemory
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 05/Oct/21 19:06
Affects Version/s: 3.1.0

TaskMemoryManager.acquireExecutionMemory has a bit of redundancy - code is duplicated in the self-spilling case. It would be good to reduce the duplication to make this more maintainable.
Bug: SampleExec can set outputOrdering as children's outputOrdering
Status: Open | Priority: Major | Resolution: 
Created: 18/Oct/21 06:55
Affects Version/s: 3.2.0

All of the code paths in SampleExec that I can see preserve the child ordering, but Spark does not set this.

Is this better to set?

 override def outputOrdering: Seq[SortOrder] = child.outputOrdering
Bug: Spark evicts RDD partitions instead of allowing OOM
Status: Open | Priority: Major | Resolution: 
Created: 09/Oct/21 11:52
Affects Version/s: 1.6.0

In the past Spark (pre 1.6) jobs would give OOM if an RDD could not fit into memory (when trying to cache with MEMORY_ONLY). These days it seems Spark jobs will evict partitions from the cache and recompute them from scratch.

We have some jobs that cache an RDD then traverse it 300 times.  In order to know when we need to increase the memory on our cluster, we need to know when it has run out of memory.

The new behaviour of Spark makes this difficult ... rather than the job OOMing (like in the past), the job instead just takes forever (our surrounding logic eventually times out the job).  Diagnosing why the job failed becomes difficult because it's not immediately obvious from the logs that the job has run out of memory (since no OOM is thrown).  One can find "evicted" log lines.

As a bit of a hack, we are using accumulators with mapPartitionsWithIndex and updating a count of the number of times each partition is traversed, then we forcibly blow up the job when this count is 2 or more (indicating an eviction).  This hack doesn't work very well as it seems to give false positives (RCA not yet understood, it doesn't seem to be speculative execution, nor are tasks failing (`egrep "task .* in stage [0-9]+\.1" | wc -l` gives 0).

*Question 1*: Is there a way to disable this new behaviour of Spark and make it behave like it used to (i.e. just blow up with OOM) - I've looked in the Spark configuration and cannot find anything like "disable-eviction".

*Question 2*: Is there any method on the SparkSession or SparkContext that we can call to easily detect when eviction is happening?

If not, then for our use case this is an effective regression - we need a way to make Spark behave predictably, or at least a way to determine automatically when Spark is running slowly due to lack of memory.


FOR CONTEXT

> Borrowed storage memory may be evicted when memory pressure arises

From https://issues.apache.org/jira/secure/attachment/12765646/unified-memory-management-spark-10000.pdf, which is attached to https://issues.apache.org/jira/browse/SPARK-10000 which was implemented in 1.6 https://spark.apache.org/releases/spark-release-1-6-0.html
Bug: JDBC option "sessionInitStatement" is ignored in resolveTable
Status: Open | Priority: Major | Resolution: 
Created: 15/Oct/21 17:00
Affects Version/s: 3.0.2

If {{sessionInitStatement}} is required to grant permissions or resolve an ambiguity, schema resolution will fail when reading a JDBC table.

Consider the following example running against Oracle database:

{code:scala}
reader.format("jdbc").options(
  Map(
    "url" -> jdbcUrl,
    "dbtable" -> "FOO",
    "user" -> "BOB",
    "sessionInitStatement" -> """ALTER SESSION SET CURRENT_SCHEMA = "BAR"""",
    "password" -> password
  )).load
{code}

Table {{FOO}} is in schema {{BAR}}, but default value for {{CURRENT_SCHEMA}} for the JDBC connection will be {{BOB}}. Therefore, the code above will fail with an error ({{ORA-00942: table or view does not exist}} if it's Oracle). It happens because [resolveTable |https://github.com/apache/spark/blob/9d061e3939a021c602c070fc13cef951a8f94c82/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala#L67]
that is called during planning phase ignores {{sessionInitStatement}}.

This might sound like an artificial example, in a simple case like the one above it's easy enough to specify {{"BOB.FOO"}} as {{dbtable}}. But when {{sessionInitStatement}} contains a more complicated setup it might not be as straightforward.
Bug: Insert support query with CTE
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/Oct/21 04:23
Affects Version/s: 3.1.2

INSERT INTO t_delta (WITH v1(c1) as (values (1)) select 1, 2,3 from v1);  OK
INSERT INTO t_delta WITH v1(c1) as (values (1)) select 1, 2,3 from v1; FAIL
Bug: pyspark os.getenv('SPARK_YARN_STAGING_DIR') can not get job path
Status: Open | Priority: Major | Resolution: 
Created: 14/Oct/21 07:25
Affects Version/s: 2.4.4

when  i submit a spark job , use spark-submit and set option --files file1, 

then in python code, i use 
{code:java}
//代码占位符
path = str(os.environ["SPARK_YARN_STAGING_DIR"])
{code}
return path is None, but this can success in java code
{code:java}
//代码占位符
spark.read().textFile(System.getenv("SPARK_YARN_STAGING_DIR") + "/README.md")
{code}
which cause this problem.
Bug: Fix json_tuple throw NPE if fields exist no foldable null value
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 13/Oct/21 04:44
Affects Version/s: 3.0.3

If json_tuple exists no foldable null field, Spark would throw NPE during eval field.toString.

e.g. the query will fail with:

{code:java}
SELECT json_tuple('{"a":"1"}', if(c1 < 1, null, 'a')) FROM ( SELECT rand() AS c1 );
{code}

{code:java}
Caused by: java.lang.NullPointerException
	at org.apache.spark.sql.catalyst.expressions.JsonTuple.$anonfun$parseRow$2(jsonExpressions.scala:435)
	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
	at scala.collection.TraversableLike.map(TraversableLike.scala:286)
	at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
	at scala.collection.AbstractTraversable.map(Traversable.scala:108)
	at org.apache.spark.sql.catalyst.expressions.JsonTuple.parseRow(jsonExpressions.scala:435)
	at org.apache.spark.sql.catalyst.expressions.JsonTuple.$anonfun$eval$6(jsonExpressions.scala:413)

{code}
Bug: Long columns cannot read columns with INT32 type in the parquet file
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 12/Oct/21 19:43
Affects Version/s: 3.1.2

The code below does not work on both Spark 3.1 and Spark 3.2.

Part of the issue is the fact that the fileSchema has logicalTypeAnnotation == null ([https://github.com/apache/spark/blob/5013171fd36e6221a540c801cb7fd9e298a6b5ba/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java#L92)] which makes isUnsignedTypeMatched return false always:

[https://github.com/apache/spark/blob/5b2f1912280e7a5afb92a96b894a7bc5f263aa6e/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java#L180] 

I am not sure even if logicalTypeAnnotation would not be null if isUnsignedTypeMatched is supposed to return true for this use case.

Python repro:
{code:java}
import os
from pyspark.sql.functions import *
from pyspark.sql import SparkSession
from pyspark.sql.types import *
spark = SparkSession.builder \
    .config("spark.hadoop.fs.s3.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem") \
    .config("spark.hadoop.fs.AbstractFileSystem.s3.impl", "org.apache.hadoop.fs.s3a.S3A") \
    .getOrCreate()
df = spark.createDataFrame([(1,2),(2,3)],StructType([StructField("id",IntegerType(),True),StructField("id2",IntegerType(),True)])).select("id")
df.write.mode("overwrite").parquet("s3://bucket/test")
df=spark.read.schema(StructType([StructField("id",LongType(),True)])).parquet("s3://bucket/test")
df.show(1, False)
{code}
Bug: Fix CREATE TABLE AS SELECT of ANSI intervals
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/Oct/21 18:37
Affects Version/s: 3.3.0

The given SQL should work:

{code:sql}
spark-sql> CREATE TABLE tbl1 STORED AS PARQUET AS SELECT INTERVAL '1-1' YEAR TO MONTH AS YM;
21/10/07 21:35:59 WARN SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
Error in query: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: Error: type expected at the position 0 of 'interval year to month' but 'interval year to month' is found
{code}
Bug: fixing "SQL column nullable setting not retained as part of spark read" issue
Status: In Progress | Priority: Major | Resolution: 
Created: 13/Oct/21 09:18
Affects Version/s: 3.0.0

Sql 'nullable' columns are not retaining 'nullable' type as it is while reading from Spark read using jdbc format.

 

SQL :

------------

 

mysql> CREATE TABLE Persons(Id int NOT NULL, FirstName varchar(255), LastName varchar(255), Age int);

 

mysql> desc Persons;
+-----------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+-------+
| Id | int | NO | | NULL | |
| FirstName | varchar(255) | YES | | NULL | |
| LastName | varchar(255) | YES | | NULL | |
| Age | int | YES | | NULL | |
+-----------+--------------+------+-----+---------+-------+

 

But in Spark  we get all the columns as "Nullable":

=============

scala> val df = spark.read.format("jdbc").option("database","Test_DB").option("user", "root").option("password", "").option("driver", "com.mysql.cj.jdbc.Driver").option("url", "jdbc:mysql://localhost:3306/Test_DB").option("dbtable", "Persons").load()
df: org.apache.spark.sql.DataFrame = [Id: int, FirstName: string ... 2 more fields]

scala> df.printSchema()
root
 |-- Id: integer (nullable = true)
 |-- FirstName: string (nullable = true)
 |-- LastName: string (nullable = true)
 |-- Age: integer (nullable = true)

=============
Bug: ignoreCorruptFiles does not work when schema change from int to string when a file having more than X records
Status: Open | Priority: Major | Resolution: 
Created: 12/Oct/21 06:24
Affects Version/s: 2.4.8

Precondition:

Spark 3.1 run locally on my Macbook Pro(16G Ram,i7, 2015)

In folder A having two parquet files
 * File 1: have some columns and one of them is column C1 with data type Int and have only one record
 * File 2: Same schema with File 1 except column C1  having data type String and having>= X records

X depends on the capacity of your computer, my case is 36, you can increase the number of row to find X.

Read file 1 to get schema of file 1.

Read folder A with schema of file 1.

Expected: Read successfully, file 2 will be ignored as the data type of column C1 changed to string.

Actual: File 2 seems to be not ignored and get error:

 `WARN TaskSetManager: Lost task 0.0 in stage 2.0 (TID 2) (192.168.1.78 executor driver): java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary WARN TaskSetManager: Lost task 0.0 in stage 2.0 (TID 2) (192.168.1.78 executor driver): java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary at org.apache.parquet.column.Dictionary.decodeToInt(Dictionary.java:45)`

 

If i remove one record from file2. It works well  
{code:java}
spark.conf.set('spark.sql.files.ignoreCorruptFiles', True)
schema1 = StructType([
 StructField("program_sk", IntegerType(), True),
 StructField("client_sk", IntegerType(), True),
])

sample_data = [(1, 17)]
df1 = spark.createDataFrame(sample_data, schema1)

schema2 = StructType([
 StructField("program_sk", IntegerType(), True),
 StructField("client_sk", StringType(), True),
])
sample_data = [(1, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 (2, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 (3, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 (4, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 (5, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 (6, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 (7, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 (8, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 (9, "19999"), (2, "3332"), (3, "199999"), (4, "3333"),
 ]
df2 = spark.createDataFrame(sample_data, schema2)
file_save_path = 's3://xxx-data-dev/adp_data_lake/test_ignore_corrupt/'

df1.write \
 .mode('overwrite') \
 .format('parquet') \
 .save(f'{file_save_path}')

df2.write \
 .mode('append') \
 .format('parquet') \
 .save(f'{file_save_path}')

df = spark.read.schema(schema1).parquet(file_save_path)
df.show(){code}
Bug: Manual disabled format `B` for `date_format` function to compatibility with Java 8 behavior.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 11/Oct/21 03:37
Affects Version/s: 3.3.0

The `date_format` function has some behavioral differences when using JDK 8 and JDK 17 as following:

the result of {{select date_format('2018-11-17 13:33:33.333', 'B')}} in {{datetime-formatting-invalid.sql}} with Java 8 is:
{code:java}
-- !query
select date_format('2018-11-17 13:33:33.333', 'B')
-- !query schema
struct<>
-- !query output
java.lang.IllegalArgumentException
Unknown pattern letter: B
{code}
and with Java 17 the result is:
{code:java}
- datetime-formatting-invalid.sql *** FAILED ***
  datetime-formatting-invalid.sql
  Expected "struct<[]>", but got "struct<[date_format(2018-11-17 13:33:33.333, B):string]>" Schema did not match for query #34
  select date_format('2018-11-17 13:33:33.333', 'B'): -- !query
  select date_format('2018-11-17 13:33:33.333', 'B')
  -- !query schema
  struct<date_format(2018-11-17 13:33:33.333, B):string>
  -- !query output
  in the afternoon (SQLQueryTestSuite.scala:469)
{code}
 
From the javadoc we can find that 'B' is used to represent `{{Pattern letters to output a day period`}} in Java 17.
Bug: Make HiveSerDe.serdeMap extensible
Status: Open | Priority: Major | Resolution: 
Created: 09/Oct/21 02:51
Affects Version/s: 2.4.9

Currently, Only tables with certain types (e.g. json, orc, parquet, avro) which defined in HiveSerDe.serdeMap can be created correctly in Hive metastore.

Other tables won't be persisted in a Hive compatible way. Only persisting these tables into Hive metastore in Spark SQL specific format as follow:
{code:java}
col    array<string>    from deserializer  
{code}
So, can we make this extensible, so that we can pass the serde info needed to spark and create a hive-compatible table by spark-sql.
Bug: Misuse "merge schema" when mapGroups
Status: Open | Priority: Major | Resolution: 
Created: 05/Oct/21 09:26
Affects Version/s: 3.0.0

{code:java}
// Test case for this bug
val spark = SparkSession.builder().master("local[*]").getOrCreate()

val data1 = Seq(
  Row("0", 1),
  Row("0", 2))
val schema1 = StructType(List(
  StructField("col0", StringType),
  StructField("col1", IntegerType))
)
val data2 = Seq(
  Row("0", 1),
  Row("0", 2))
val schema2 = StructType(List(
  StructField("str0", StringType),
  StructField("col0", IntegerType))
)

val df1 = spark.createDataFrame(spark.sparkContext.makeRDD(data1), schema1)
val df2 = spark.createDataFrame(spark.sparkContext.makeRDD(data2), schema2)

val joined = df1.join(df2, df1("col0") === df2("str0"), "left")

import spark.implicits._
val distinct = joined
  .groupByKey {
    row => row.getInt(1)
  }
  .mapGroups {
    case (_, iter) =>
        iter.maxBy(row => {
          row.getInt(3)
        })
  }(RowEncoder(joined.schema))

distinct.show(){code}
{code:java}
 // A part of errors
Exception in thread "main" org.apache.spark.SparkException: Failed to merge fields 'col0' and 'col0'. Failed to merge incompatible data types string and int 
at org.apache.spark.sql.types.StructType$.$anonfun$merge$2(StructType.scala:593)
at scala.Option.map(Option.scala:163)
at org.apache.spark.sql.types.StructType$.$anonfun$merge$1(StructType.scala:585)
at org.apache.spark.sql.types.StructType$.$anonfun$merge$1$adapted
(StructType.scala:582)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198) 
at org.apache.spark.sql.types.StructType$.merge(StructType.scala:582) 
at org.apache.spark.sql.types.StructType.merge(StructType.scala:492)
at org.apache.spark.sql.catalyst.expressions.SchemaPruning$.$
anonfun$pruneDataSchema$2(SchemaPruning.scala:36)
at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122) 
at scala.collection.immutable.List.foldLeft(List.scala:89) 
at scala.collection.LinearSeqOptimized.reduceLeft(LinearSeqOptimized.scala:140) 
at scala.collection.LinearSeqOptimized.reduceLeft$(LinearSeqOptimized.scala:138) 
at scala.collection.immutable.List.reduceLeft(List.scala:89)
{code}

After left join two dataframe which have two shemas with the same name but different types, we use groupByKey and mapGroups to get the result. But it will makes some mistakes. Is it my grammatical mistake? If not, I think It may be related to schema merge in StructType.scala: 593. How can I turn off schema merging?
Bug: Improve error message for missing column
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/Oct/21 00:17
Affects Version/s: 3.3.0

Improve the error message for the case that a user asks for a column that does not exist.
Today, the message is "cannot resolve 'foo' given input columns [bar, baz, froo]".
We should sort the suggestion list by similarity and improve the grammar to remove lingo like "resolve."
Bug: Renamed columns of parquet tables become NULL
Status: Open | Priority: Major | Resolution: 
Created: 08/Oct/21 17:22
Affects Version/s: 3.0.3

If a column of a Parquet table gets renamed in Hive metastore, then Spark (SQL) is unable to read the relevant values, and all of the renamed column becomes NULL.

The problem could be reproduced with the following SQL queries:

 
{noformat}
create table tmp.parquet_table1 (i1 int, s1 string) stored as parquet;
insert into tmp.parquet_table1 values (1, "AAA"), (2, "BBB"), (3, "CCC");
select * from tmp.parquet_table1;
+---+---+
| i1| s1|
+---+---+
| 1|AAA|
| 2|BBB|
| 3|CCC|
+---+---+
alter table tmp.parquet_table1 replace columns (i2 int, s1 string);
select * from tmp.parquet_table1;
+----+---+
| i2| s1|
+----+---+
|null|AAA|
|null|BBB|
|null|CCC|
+----+---+ 
{noformat}
Notice, that column `i1` is renamed to `i2`, which became NULL afterwards. 

{{I have used Impala to create the table, and insert the values, but Spark (SQL) could be used as well. "alter table ... replace columns ..." could only be executed outside of Spark (Impala or Hive).}}

{{No [configuration option|https://spark.apache.org/docs/latest/configuration.html] helps, that I am aware of.}}

{{With Spark 2.4.8, this works correctly.}}
Bug: Exception when trying to access Row field using getAs method
Status: Open | Priority: Major | Resolution: 
Created: 07/Oct/21 15:03
Affects Version/s: 3.1.2

I have an input dataframe *df* with the following schema:
{code:java}
|-- origin: string (nullable = true)
|-- product: struct (nullable = true)
|    |-- id: integer (nullable = true){code}
 

when I try to select the first 20 rows of the id column I execute:
{code:java}
df.select("product.id").show(20, false)
{code}
 

and I manage to get the result. But when I execute the following: 
{code:java}
df.map(_.getAs[Int]("product.id")).show(20, false){code}
 

I get the following error:
{code:java}
java.lang.IllegalArgumentException: Field "product.id" does not exist.{code}
Bug: Inline type hints for python/pyspark/sql/avro/functions.py
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Oct/21 18:03
Affects Version/s: 3.3.0

Inline type hints for python/pyspark/sql/avro/functions.py.

 

Currently, we use stub files for type annotations, which don't support type checks within function bodies. So we inline type hints to support that.
Bug: unionByName shouldn't consider types when comparing structs
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/Oct/21 01:13
Affects Version/s: 3.2.0

Improvement/follow-on of https://issues.apache.org/jira/browse/SPARK-35290.

We use StructType.sameType to see if we need to recreate the struct, but this can lead to false positives if the structure is the same but the types are different, and will lead to simply creating a new struct that's exactly the same as the original. This can cause significant overhead when unioning multiple deeply nested nullable structs, as each time it's recreated it gets wrapped in a If(IsNull()). Only comparing the field names can lead to more efficient plans.
Bug: Add AQE Planning Times to SQL Metrics
Status: In Progress | Priority: Major | Resolution: 
Created: 01/Oct/21 22:28
Affects Version/s: 3.1.2

Add metrics for durations of "reOptimize", "generate explainString" and "createQueryStages" to AdaptiveSparkPlanExec metrics to make it easier to see overhead of AQE for a query
Bug: Enable Dependabot for improving security posture of the dependencies
Status: In Progress | Priority: Major | Resolution: 
Created: 03/Oct/21 19:27
Affects Version/s: 3.1.2

h3. Why are the changes needed?

[https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependencies-updated-automatically]

Having knowledge about vulnerabilities of the dependencies helps the project owners decide on their dependencies security posture to make decisions.

If the project decides to get updates only on security updates and not on any version updates then setting these options would not open any PR 's {{open-pull-requests-limit: 0}}
Bug: Implement StructType.toString explicitly for Scala 2.13
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 20/Oct/21 17:53
Affects Version/s: 3.3.0

The string returned by StructType.toString is different  between Scala 2.12 and 2.13.

* Scala 2.12
{code}
val st = StructType(StructField("a", IntegerType) :: Nil)
st.toString
res0: String = StructType(StructField(a,IntegerType,true)
{code}

* Scala 2.13
{code}
val st = StructType(StructField("a", IntegerType) :: Nil)
st.toString
val res0: String = Seq(StructField(a,IntegerType,true))
{code}

It's because the logic to make the prefix of the string was changed from Scala 2.13.

Scala 2.12: https://github.com/scala/scala/blob/v2.12.15/src/library/scala/collection/TraversableLike.scala#L804
Scala 2:13:https://github.com/scala/scala/blob/v2.13.5/src/library/scala/collection/Seq.scala#L46