av9ash/GitBugs huggingface dataset:
Bug: Null Pointer Exeption on unpersist call
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 05/May/22 07:09
Affects Version/s: 3.2.1

DataFrame.unpesist call fails wth NPE

 
{code:java}
java.lang.NullPointerException
    at org.apache.spark.sql.execution.columnar.CachedRDDBuilder.isCachedRDDLoaded(InMemoryRelation.scala:247)
    at org.apache.spark.sql.execution.columnar.CachedRDDBuilder.isCachedColumnBuffersLoaded(InMemoryRelation.scala:241)
    at org.apache.spark.sql.execution.CacheManager.$anonfun$uncacheQuery$8(CacheManager.scala:189)
    at org.apache.spark.sql.execution.CacheManager.$anonfun$uncacheQuery$8$adapted(CacheManager.scala:176)
    at scala.collection.TraversableLike.$anonfun$filterImpl$1(TraversableLike.scala:304)
    at scala.collection.Iterator.foreach(Iterator.scala:943)
    at scala.collection.Iterator.foreach$(Iterator.scala:943)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
    at scala.collection.TraversableLike.filterImpl(TraversableLike.scala:303)
    at scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:297)
    at scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108)
    at scala.collection.TraversableLike.filter(TraversableLike.scala:395)
    at scala.collection.TraversableLike.filter$(TraversableLike.scala:395)
    at scala.collection.AbstractTraversable.filter(Traversable.scala:108)
    at org.apache.spark.sql.execution.CacheManager.recacheByCondition(CacheManager.scala:219)
    at org.apache.spark.sql.execution.CacheManager.uncacheQuery(CacheManager.scala:176)
    at org.apache.spark.sql.Dataset.unpersist(Dataset.scala:3220)
    at org.apache.spark.sql.Dataset.unpersist(Dataset.scala:3231){code}
Looks like syncronization in required for org.apache.spark.sql.execution.columnar.CachedRDDBuilder#isCachedColumnBuffersLoaded

 
{code:java}
def isCachedColumnBuffersLoaded: Boolean = {
  _cachedColumnBuffers != null && isCachedRDDLoaded
}

def isCachedRDDLoaded: Boolean = {
    _cachedColumnBuffersAreLoaded || {
      val bmMaster = SparkEnv.get.blockManager.master
      val rddLoaded = _cachedColumnBuffers.partitions.forall { partition =>
        bmMaster.getBlockStatus(RDDBlockId(_cachedColumnBuffers.id, partition.index), false)
          .exists { case(_, blockStatus) => blockStatus.isCached }
      }
      if (rddLoaded) {
        _cachedColumnBuffersAreLoaded = rddLoaded
      }
      rddLoaded
  }
} {code}
isCachedRDDLoaded relies on _cachedColumnBuffers != null check while it can be changed concurrently from other thread.
Bug: Log cost time for Get FileStatus in HadoopTableReader
Status: In Progress | Priority: Major | Resolution: 
Created: 09/May/22 04:46
Affects Version/s: 3.3.0

Sometimes fs.globStatus(pathPattern) costs unreasonable time, but there is not log for recording.

!image-2022-05-09-12-43-38-906.png!

￼ !image-2022-05-09-12-44-27-288.png!
Bug: Stream is corrupted Exception while fetching the blocks from fallback storage system
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/May/22 21:29
Affects Version/s: 3.2.0

When executor decommissioning and fallback storage is enabled - the shuffle reads are failing with `FetchFailedException: Stream is corrupted` 

ref: https://issues.apache.org/jira/browse/SPARK-18105 (search for decommission)

 

This is happening when the shuffle block is bigger than `inputstream.read` can read in one attempt. The code path is not reading the block fully (`readFully`) and the partial read is causing the exception.
Bug: ArrayIndexOutOfBoundsException for some date/time sequences in some time-zones
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 13/May/22 22:05
Affects Version/s: 3.1.3

The following query gets an {{ArrayIndexOutOfBoundsException}} when run from the {{America/Los_Angeles}} time-zone:
{noformat}
spark-sql> select sequence(timestamp'2022-03-13 00:00:00', timestamp'2022-03-16 03:00:00', interval 1 day 1 hour) as x;
22/05/13 14:47:27 ERROR SparkSQLDriver: Failed in [select sequence(timestamp'2022-03-13 00:00:00', timestamp'2022-03-16 03:00:00', interval 1 day 1 hour) as x]
java.lang.ArrayIndexOutOfBoundsException: 3
{noformat}
In fact, any such query will get an {{ArrayIndexOutOfBoundsException}} if the start-stop period in your time-zone includes more instances of "spring forward" than instances of "fall back" and the start-stop period is evenly divisible by the interval.

In the {{America/Los_Angeles}} time-zone, examples include:
{noformat}
-- This query encompasses 2 instances of "spring forward" but only one
-- instance of "fall back".
select sequence(
  timestamp'2022-03-13',
  timestamp'2022-03-13' + (interval '42' hours * 209),
  interval '42' hours) as x;
{noformat}
{noformat}
select sequence(
  timestamp'2022-03-13',
  timestamp'2022-03-13' + (interval '31' hours * 11),
  interval '31' hours) as x;
{noformat}
Bug: Make Dataset.melt work with struct fields
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/May/22 14:26
Affects Version/s: 3.4.0

In SPARK-38864, the melt function was added to Dataset.

It would be nice if fields of struct fields could be used as id and value columns. This would allow for the following:

Given a Dataset with following schema:
{code:java}
root
 |-- an: struct (nullable = false)
 |    |-- id: integer (nullable = false)
 |-- str: struct (nullable = false)
 |    |-- one: string (nullable = true)
 |    |-- two: string (nullable = true)
{code}

For example:
{code:java}
+---+-------------+
| an|          str|
+---+-------------+
|{1}|   {one, One}|
|{2}|  {two, null}|
|{3}|{null, three}|
|{4}| {null, null}|
+---+-------------+
{code}
Melting with value columns {{Seq("str.one", "str.two")}} on id columns {{Seq("an.id")}} would result in
{code:java}
+--+--------+-----+
|an|variable|value|
+--+--------+-----+
| 1| str.one|  one|
| 1| str.two|  One|
| 2| str.one|  two|
| 2| str.two| null|
| 3| str.one| null|
| 3| str.two|three|
| 4| str.one| null|
| 4| str.two| null|
+--+--------+-----+
{code}

See test in {{org.apache.spark.sql.MeltSuite}}:
{code:java}
  test("SPARK-39292: melt with struct fields") {
    val df = meltWideDataDs.select(
      struct($"id").as("an"),
      struct(
        $"str1".as("one"),
        $"str2".as("two")
      ).as("str")
    )

    checkAnswer(
      Melt.of(df, Seq("an.id"), Seq("str.one", "str.two"), false, "variable", "value"),
      meltedWideDataRows.map(row => Row(
        row.getInt(0),
        row.getString(1) match {
          case "str1" => "str.one"
          case "str2" => "str.two"
        },
        row.getString(2)
      ))
    )
  }
{code}
Bug: Improve error messages: step 1
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/May/22 08:13
Affects Version/s: 3.4.0

Improve error messages of the following error classes:
 # [INVALID_JSON_SCHEMA_MAP_TYPE|https://github.com/apache/spark/commit/2183ceeb09b9217067d215b0173f75ddaaea474c]
 # [INCOMPARABLE_PIVOT_COLUMN|https://github.com/apache/spark/commit/cbfae161f4a9255ff4e20482d9385b90322d8dc6]
 # [INVALID_ARRAY_INDEX_IN_ELEMENT_AT|https://github.com/apache/spark/commit/ac462de4fa169af2146d8fd5e9f82b86c38524e1]
 # [INVALID_ARRAY_INDEX|https://github.com/apache/spark/commit/6bb07183d56296c6eeeef39b7d5fe5ebf46a494e]
 # [DIVIDE_BY_ZERO|https://github.com/apache/spark/commit/2532b51f9cb66d44ed63fcfb3554cbbf3815402a]
Bug: Improve error messages: step 2
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 23/May/22 08:38
Affects Version/s: 3.4.0

Improve the following error messages:
1. NON_PARTITION_COLUMN
2. UNSUPPORTED_SAVE_MODE
3. INVALID_FIELD_NAME
4. FAILED_SET_ORIGINAL_PERMISSION_BACK
5. NON_LITERAL_PIVOT_VALUES
6. INVALID_SYNTAX_FOR_CAST
Bug: Reach parity with pandas boolean cast
Status: Open | Priority: Major | Resolution: 
Created: 18/May/22 18:34
Affects Version/s: 3.4.0

There are pandas APIs that need boolean casts: all, any.

Currently, pandas-on-Spark has different behaviors on special inputs against these APIs, for example, empty string, list, etc, as mentioned https://github.com/apache/spark/pull/36547#issuecomment-1129228724 by [~zero323].

We shall match pandas behavior on boolean cast.

Meanwhile, Series/Frame that contains empty strings, lists should be considered as test input to increase test coverage.
Bug: StreamCorruptedException cause job failure for disk persisted RDD
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 11/May/22 18:02
Affects Version/s: 3.4.0

In case of a disk corruption a disk persisted RDD block will lead to job failure as the block registration is always leads to the same file. So even when the task is rescheduled on a different executor the job will fail.

*Example*

First failure (the block is locally available):
{noformat}
22/04/25 07:15:28 ERROR executor.Executor: Exception in task 17024.0 in stage 12.0 (TID 51853)
java.io.StreamCorruptedException: invalid stream header: 00000000
  at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:943)
  at java.io.ObjectInputStream.<init>(ObjectInputStream.java:401)
  at org.apache.spark.serializer.JavaDeserializationStream$$anon$1.<init>(JavaSerializer.scala:63)
  at org.apache.spark.serializer.JavaDeserializationStream.<init>(JavaSerializer.scala:63)
  at org.apache.spark.serializer.JavaSerializerInstance.deserializeStream(JavaSerializer.scala:122)
  at org.apache.spark.serializer.SerializerManager.dataDeserializeStream(SerializerManager.scala:209)
  at org.apache.spark.storage.BlockManager.getLocalValues(BlockManager.scala:617)
  at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:897)
  at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:335)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:286)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
{noformat}

Then the task might be rescheduled on a different executor but as the block is registered to the first blockmanager the error will be the same:

{noformat}
java.io.StreamCorruptedException: invalid stream header: 00000000
  at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:943)
  at java.io.ObjectInputStream.<init>(ObjectInputStream.java:401)
  at org.apache.spark.serializer.JavaDeserializationStream$$anon$1.<init>(JavaSerializer.scala:63)
  at org.apache.spark.serializer.JavaDeserializationStream.<init>(JavaSerializer.scala:63)
  at org.apache.spark.serializer.JavaSerializerInstance.deserializeStream(JavaSerializer.scala:122)
  at org.apache.spark.serializer.SerializerManager.dataDeserializeStream(SerializerManager.scala:209)
  at org.apache.spark.storage.BlockManager$$anonfun$getRemoteValues$1.apply(BlockManager.scala:698)
  at org.apache.spark.storage.BlockManager$$anonfun$getRemoteValues$1.apply(BlockManager.scala:696)
  at scala.Option.map(Option.scala:146)
  at org.apache.spark.storage.BlockManager.getRemoteValues(BlockManager.scala:696)
  at org.apache.spark.storage.BlockManager.get(BlockManager.scala:831)
  at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:886)
  at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:335)
{noformat}

My idea is to retry the IO operations a few times and when all of them failed deregistering the block and let the following task to recompute it.
Bug: Throw an exception w/ error class for an invalid bucket file
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/May/22 10:03
Affects Version/s: 3.4.0

Replace IllegalStateException by Spark's exception w/ an error class there [https://github.com/apache/spark/blob/ee6ea3c68694e35c36ad006a7762297800d1e463/sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala#L621.|https://github.com/apache/spark/blob/ee6ea3c68694e35c36ad006a7762297800d1e463/sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala#L621]

Move related tests to an Query.*ErrorsSuite.
Bug: Timestamps returned by now() and equivalent functions are not consistent in subqueries
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 23/May/22 16:04
Affects Version/s: 3.2.1

Timestamp evaluation in not consistent across subqueries. As an example for the Spark Shell

 
{code:java}
sql("SELECT * FROM (SELECT 1) WHERE now() IN (SELECT now())").collect() {code}
Returns an empty result.

 

The root cause is that [ComputeCurrentTime|https://github.com/apache/spark/blob/c91c2e9afec0d5d5bbbd2e155057fe409c5bb928/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala#L74] does not iterate into subqueries
Bug: Improve MapOutputTracker convertMapStatuses performance
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 28/May/22 06:45
Affects Version/s: 3.4.0

How to reproduce this issue:
{code:java}
val benchmark = new Benchmark("MapStatuses Convert", 1, output = output)

val blockManagerNumber = 1000
val mapNumber = 50000
val shufflePartitions = 10000

val shuffleId: Int = 0
// First reduce task will fetch map data from startPartition to endPartition
val startPartition = 0
val startMapIndex = 0
val endMapIndex = mapNumber
val blockManagers = Array.tabulate(blockManagerNumber) { i =>
  BlockManagerId("a", "host" + i, 7337)
}
val mapStatuses: Array[MapStatus] = Array.tabulate(mapNumber) { mapTaskId =>
  HighlyCompressedMapStatus(
    blockManagers(mapTaskId % blockManagerNumber),
    Array.tabulate(shufflePartitions)(i => if (i % 50 == 0) 1 else 0),
    mapTaskId)
}
val bitmap = new RoaringBitmap()
Range(0, 4000).foreach(bitmap.add(_))
val mergeStatuses = Array.tabulate(shufflePartitions) { part =>
  MergeStatus(blockManagers(part % blockManagerNumber), shuffleId, bitmap, 100)
}

Array(499, 999, 1499).foreach { endPartition =>
  benchmark.addCase(
    s"Num Maps: $mapNumber Fetch partitions:${endPartition - startPartition + 1}",
    numIters) { _ =>
    MapOutputTracker.convertMapStatuses(
      shuffleId,
      startPartition,
      endPartition,
      mapStatuses,
      startMapIndex,
      endMapIndex,
      Some(mergeStatuses))
  }
}

benchmark.run() {code}
Benchmark result
{code:java}
================================================================================================ MapStatuses Convert Benchmark ================================================================================================Java HotSpot(TM) 64-Bit Server VM 1.8.0_281-b09 on Mac OS X 10.15.7 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz MapStatuses Convert:                      Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative ------------------------------------------------------------------------------------------------------------------------ Num Maps: 50000 Fetch partitions:500               3393           3483          96          0.0  3393439257.0       1.0X Num Maps: 50000 Fetch partitions:1000              6640           6772         121          0.0  6639654832.0       0.5X Num Maps: 50000 Fetch partitions:1500             10035          10143         108          0.0 10035100069.0       0.3X
{code}
Bug: Make CreateTable API and ListTables API compatible 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/May/22 19:29
Affects Version/s: 3.3.0

https://github.com/apache/spark/blob/c6dccc7dd412a95007f5bb2584d69b85ff9ebf8e/sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala#L364

https://github.com/apache/spark/blob/c6dccc7dd412a95007f5bb2584d69b85ff9ebf8e/sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala#L99
Bug: Add a CheckError() method to SparkFunSuite
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/May/22 16:01
Affects Version/s: 3.3.1

We want to standardize on a generic way to QA error messages without impeding the ability to enhance/rework error messages.
CheckError() allows for efficient asserting on the "payload":
 * Errorclass, subclass
 * SQLState
 * Parameters (both names and values)

 

It does not test the actual English text. Which is the feature
Bug: Refactor DescribeTableExec
Status: Resolved | Priority: Major | Resolution: Later
Created: 30/May/22 05:05
Affects Version/s: 3.3.0

Repeated code, refactor the code.


{code:java}

  private def addTableDetails(rows: ArrayBuffer[InternalRow]): Unit = {
    rows += emptyRow()
    rows += toCatalystRow("# Detailed Table Information", "", "")
    rows += toCatalystRow("Name", table.name(), "")

    CatalogV2Util.TABLE_RESERVED_PROPERTIES.foreach(propKey => {
      if (table.properties.containsKey(propKey)) {
        rows += toCatalystRow(propKey.capitalize, table.properties.get(propKey), "")
      }
    })
    val properties =
      table.properties.asScala.toList
        .filter(kv => !CatalogV2Util.TABLE_RESERVED_PROPERTIES.contains(kv._1))
        .sortBy(_._1).map {
        case (key, value) => key + "=" + value
      }.mkString("[", ",", "]")
    rows += toCatalystRow("Table Properties", properties, "")
  }
{code}
Bug: Flaky Test: `INSERT rows, ALTER TABLE ADD COLUMNS with DEFAULTs, then SELECT them`
Status: Closed | Priority: Major | Resolution: Duplicate
Created: 29/May/22 23:58
Affects Version/s: 3.4.0

{code}
INSERT rows, ALTER TABLE ADD COLUMNS with DEFAULTs, then SELECT them *** FAILED *** (1 second, 297 milliseconds)
{code}

- https://github.com/apache/spark/runs/6616769543
- https://github.com/apache/spark/runs/6617574099
- https://github.com/apache/spark/runs/6619396708
- https://github.com/apache/spark/runs/6619465249
- https://github.com/apache/spark/runs/6638234468

The above flaky test succeeds locally.
{code}
$ build/sbt "sql/testOnly org.apache.spark.sql.sources.InsertSuite -- -z INSERT"
...
[info] All tests passed.
[success] Total time: 69 s (01:09), completed May 29, 2022, 5:11:06 PM
{code}
Bug: Flaky Test: 'single listener, check trigger events are generated correctly`
Status: Open | Priority: Major | Resolution: 
Created: 29/May/22 23:54
Affects Version/s: 3.4.0

{code}
StreamingQueryListenerSuite:
single listener, check trigger events are generated correctly *** FAILED *** (276 milliseconds)
{code}

- https://github.com/apache/spark/runs/6616775239
Bug: Flaky Test: `SPARK-37753: Inhibit broadcast in left outer join when there are many empty partitions on outer/left side`
Status: Open | Priority: Major | Resolution: 
Created: 29/May/22 23:51
Affects Version/s: 3.4.0

{code}
SPARK-37753: Inhibit broadcast in left outer join when there are many empty partitions on outer/left side *** FAILED *** (219 milliseconds)
{code}

- https://github.com/apache/spark/runs/6645647700
Bug: Refactor PromotePrecision and CheckOverflow with decimal binary arithmetic
Status: Open | Priority: Major | Resolution: 
Created: 27/May/22 08:21
Affects Version/s: 3.4.0

For a decimal binary arithmetic expression, we correct its runtime data type and result data type at DecimalPrecision. It has two drawback:
 # it has some bug with the different ordering of the rules in TypeCoercion
 # it's unnecessary to change its runtime data type for decimal, as it supports eval with different precision. So what we need to do is change the result type precision.
Bug: Implement the EQUAL_NULL function
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/May/22 14:37
Affects Version/s: 3.4.0

Add new function EQUAL_NULL.

*Returns TRUE:*
  EQUAL_NULL( <null> , <null> )

*Returns FALSE:*
  EQUAL_NULL( <null> , <not_null> )
  EQUAL_NULL( <not_null> , <null> )

*Otherwise:*
  EQUAL_NULL(<expr1>, <expr2>) is equivalent to <expr1> = <expr2>
Bug: Proposed spark.SQL.function to return an array of fixed sized collections from a collection (array or string)
Status: Open | Priority: Major | Resolution: 
Created: 26/May/22 21:27
Affects Version/s: 3.2.1

In scenarios where a lot of UDFs centered around chunking collections into fixed* sized sub-arrays or arrays of strings.

The only alternative is to use complex and expensive regex that does not work for non-String use cases.

*left over elements carrying the remainder of the size

Example

===

scala> def grouped_by_size[C](size: Int, collection: Iterable[C]) = collection.grouped(size).toArray
grouped_by_size: [C](size: Int, collection: Iterable[C])Array[Iterable[C]]

scala> grouped_by_size(2, "Vishal Singhrd")
res0: Array[Iterable[Char]] = Array(Vi, sh, al, S, in, gh, rd)

scala> grouped_by_size(2, (0 to 4).toSeq )
res1: Array[Iterable[Int]] = Array(Vector(0, 1), Vector(2, 3), Vector(4))

scala> grouped_by_size(3, "Vishal Singhrd")
res2: Array[Iterable[Char]] = Array(Vis, hal, Si, ngh, rd)

scala> grouped_by_size(3, Array("This","is","my","last","example"))
res3: Array[Iterable[String]] = Array(WrappedArray(This, is, my), WrappedArray(last, example))
 * Elements in an array or string, depending on what the data domain is, may have a natural periodicity that has semantic meaning.
 * Being able to easily divide a large collection into an Array of Arrays or Array of String aids in applying transforms, explodes, and other array functions (zip, etc) at the appropriate level of periodicity for the data domain.

Pros:
 * Current methods for creating periodicity require Strings only as well as complex and inefficient regular expressions instead a simpler and more direct solution. The proposed solution would work for both Strings and Arrays.
Bug: Keep the Spark SQL API reference URL as same as before.
Status: Open | Priority: Major | Resolution: 
Created: 25/May/22 12:43
Affects Version/s: 3.3.0

After https://github.com/apache/spark/pull/36647 is merged, the PySpark SQL API reference URL is changed, since the file structure is changed from `pyspark.sql.rst` to `pyspark.sql/*.rst`.

We should keep it same as before to keep the consistency with older documents versions.
Bug: Fasten Timestamp type inference of default format in JSON/CSV data source
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/May/22 08:16
Affects Version/s: 3.3.0

When reading JSON/CSV files with inferring timestamp types `.option("inferTimestamp", true)`, the Timestamp conversion will throw and catch exceptions. As we are putting decent error messages in the exception, the creation of the exceptions is actually not cheap. It consumes more than 90% of the type inference time. 

We can use the parsing methods which return optional results instead.

Before the change, it takes 166 seconds to infer a JSON file of 624MB with inferring timestamp enabled.

After the change, it only 16 seconds.
Bug: Fasten the schema inference of CSV/JSON data source
Status: Open | Priority: Major | Resolution: 
Created: 25/May/22 02:48
Affects Version/s: 3.4.0

In the current implementation of CSV/JSON data source, the schema inference relies on methods that will throw exceptions if the fields can't convert as some data types. 

Throwing and catching exceptions can be slow. We can improve it by creating methods that return optional results instead. A good example is [https://github.com/apache/spark/pull/36562|https://github.com/apache/spark/pull/36562,], which reduces the schema inference time by 90%.
Bug: Improve errors related to CAST
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/May/22 18:09
Affects Version/s: 3.3.0

1. Rename the error classes INVALID_SYNTAX_FOR_CAST and CAST_CAUSES_OVERFLOW to make more precise and clear.
2. Improve error messages of the error classes (use quotes for SQL config and function names).