av9ash/GitBugs huggingface dataset:
Bug: Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly
Status: In Progress | Priority: Minor | Resolution: 
Created: 23/Mar/22 19:28
Affects Version/s: 3.1.2

There's an existing flag "spark.sql.files.ignoreCorruptFiles" and "spark.sql.files.ignoreMissingFiles" that will quietly ignore attempted reads from files that have been corrupted, but it still allows the query to fail on sequence files.

 

Being able to ignore corrupt record is useful in the scenarios that users want to query successfully in dirty data(mixed schema in one table).

 

We would like to add a "spark.sql.hive.ignoreCorruptRecord"  to fill out the functionality.
Bug: Test the error class: INVALID_INPUT_SYNTAX_FOR_NUMERIC_TYPE
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 18:07
Affects Version/s: 3.4.0

Add a test for the error class *INVALID_INPUT_SYNTAX_FOR_NUMERIC_TYPE* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def invalidInputSyntaxForNumericError(s: UTF8String): NumberFormatException = {
    new SparkNumberFormatException(errorClass = "INVALID_INPUT_SYNTAX_FOR_NUMERIC_TYPE",
      messageParameters = Array(s.toString, SQLConf.ANSI_ENABLED.key))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: UNRECOGNIZED_SQL_TYPE
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 20:16
Affects Version/s: 3.4.0

Add a test for the error classes *UNRECOGNIZED_SQL_TYPE* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def unrecognizedSqlTypeError(sqlType: Int): Throwable = {
    new SparkSQLException(errorClass = "UNRECOGNIZED_SQL_TYPE", Array(sqlType.toString))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the pivot error classes
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 18:25
Affects Version/s: 3.4.0

Add tests for the error classes *PIVOT_VALUE_DATA_TYPE_MISMATCH* and *NON_LITERAL_PIVOT_VALUES* to QueryCompilationErrorsSuite. The test should cover the exception throw in QueryCompilationErrors:

{code:scala}
  def nonLiteralPivotValError(pivotVal: Expression): Throwable = {
    new AnalysisException(
      errorClass = "NON_LITERAL_PIVOT_VALUES",
      messageParameters = Array(pivotVal.toString))
  }

  def pivotValDataTypeMismatchError(pivotVal: Expression, pivotCol: Expression): Throwable = {
    new AnalysisException(
      errorClass = "PIVOT_VALUE_DATA_TYPE_MISMATCH",
      messageParameters = Array(
        pivotVal.toString, pivotVal.dataType.simpleString, pivotCol.dataType.catalogString))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: INCOMPATIBLE_DATASOURCE_REGISTER
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 17:56
Affects Version/s: 3.4.0

Add at least one test for the error class *INCOMPATIBLE_DATASOURCE_REGISTER* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def incompatibleDataSourceRegisterError(e: Throwable): Throwable = {
    new SparkClassNotFoundException("INCOMPATIBLE_DATASOURCE_REGISTER", Array(e.getMessage), e)
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: PySpark ResourceWarning: unclosed socket
Status: Open | Priority: Minor | Resolution: 
Created: 25/Mar/22 13:11
Affects Version/s: 3.2.1

Create a file called `spark.py` with the following contents:

```
from pyspark.sql import SparkSession

with SparkSession.builder.getOrCreate() as spark:
    spark.read.csv('test.csv').collect()
```

You can also create a `test.csv` file with whatever data in it. When executing `python -Wall spark.py` I get the following warning:

```
/usr/lib/python3.8/socket.py:740: ResourceWarning: unclosed <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 51344), raddr=('127.0.0.1', 39221)>
  self._sock = None
ResourceWarning: Enable tracemalloc to get the object allocation traceback
```
Bug: Test the error class: FAILED_SET_ORIGINAL_PERMISSION_BACK
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 17:42
Affects Version/s: 3.4.0

Add at least one test for the error class *FAILED_SET_ORIGINAL_PERMISSION_BACK* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def failToSetOriginalPermissionBackError(
      permission: FsPermission,
      path: Path,
      e: Throwable): Throwable = {
    new SparkSecurityException(errorClass = "FAILED_SET_ORIGINAL_PERMISSION_BACK",
      Array(permission.toString, path.toString, e.getMessage))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error classes: INVALID_FIELD_NAME
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 18:03
Affects Version/s: 3.4.0

Add tests for the error class *INVALID_FIELD_NAME* to QueryCompilationErrorsSuite. The test should cover the exception throw in QueryCompilationErrors:

{code:scala}
  def invalidFieldName(fieldName: Seq[String], path: Seq[String], context: Origin): Throwable = {
    new AnalysisException(
      errorClass = "INVALID_FIELD_NAME",
      messageParameters = Array(fieldName.quoted, path.quoted),
      origin = context)
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: AMBIGUOUS_FIELD_NAME
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 16:35
Affects Version/s: 3.4.0

Add at least one test for the error class *AMBIGUOUS_FIELD_NAME* to QueryCompilationErrorsSuite. The test should cover the exception throw in QueryCompilationErrors:

{code:scala}
  def ambiguousFieldNameError(
      fieldName: Seq[String], numMatches: Int, context: Origin): Throwable = {
    new AnalysisException(
      errorClass = "AMBIGUOUS_FIELD_NAME",
      messageParameters = Array(fieldName.quoted, numMatches.toString),
      origin = context)
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: MAP_KEY_DOES_NOT_EXIST*
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 18:11
Affects Version/s: 3.4.0

Add tests for the error classes *MAP_KEY_DOES_NOT_EXIST** to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def mapKeyNotExistError(key: Any, isElementAtFunction: Boolean): NoSuchElementException = {
    if (isElementAtFunction) {
      new SparkNoSuchElementException(errorClass = "MAP_KEY_DOES_NOT_EXIST_IN_ELEMENT_AT",
        messageParameters = Array(key.toString, SQLConf.ANSI_ENABLED.key))
    } else {
      new SparkNoSuchElementException(errorClass = "MAP_KEY_DOES_NOT_EXIST",
        messageParameters = Array(key.toString, SQLConf.ANSI_STRICT_INDEX_OPERATOR.key))
    }
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: INVALID_JSON_SCHEMA_MAPTYPE
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 18:08
Affects Version/s: 3.4.0

Add a test for the error class *INVALID_JSON_SCHEMA_MAPTYPE* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def invalidJsonSchema(schema: DataType): Throwable = {
    new AnalysisException(
      errorClass = "INVALID_JSON_SCHEMA_MAPTYPE",
      messageParameters = Array(schema.toString))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: SECOND_FUNCTION_ARGUMENT_NOT_INTEGER
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 20:14
Affects Version/s: 3.4.0

Add a test for the error classes *SECOND_FUNCTION_ARGUMENT_NOT_INTEGER* to QueryCompilationErrorsSuite. The test should cover the exception throw in QueryCompilationErrors:

{code:scala}
  def secondArgumentOfFunctionIsNotIntegerError(
      function: String, e: NumberFormatException): Throwable = {
    // The second argument of '{function}' function needs to be an integer
    new AnalysisException(
      errorClass = "SECOND_FUNCTION_ARGUMENT_NOT_INTEGER",
      messageParameters = Array(function),
      cause = Some(e))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: INCOMPARABLE_PIVOT_COLUMN
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 17:54
Affects Version/s: 3.4.0

Add at least one test for the error class *INCOMPARABLE_PIVOT_COLUMN* to QueryCompilationErrorsSuite. The test should cover the exception throw in QueryCompilationErrors:

{code:scala}
  def unorderablePivotColError(pivotCol: Expression): Throwable = {
    new AnalysisException(
      errorClass = "INCOMPARABLE_PIVOT_COLUMN",
      messageParameters = Array(pivotCol.toString))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error classes: INVALID_ARRAY_INDEX*
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 18:02
Affects Version/s: 3.4.0

Add tests for the error classes *INVALID_ARRAY_INDEX* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  private def invalidArrayIndexErrorInternal(
      index: Int,
      numElements: Int,
      key: String): ArrayIndexOutOfBoundsException = {
    new SparkArrayIndexOutOfBoundsException(errorClass = "INVALID_ARRAY_INDEX",
      messageParameters = Array(index.toString, numElements.toString, key))
  }

  def invalidElementAtIndexError(
       index: Int,
       numElements: Int): ArrayIndexOutOfBoundsException = {
    new SparkArrayIndexOutOfBoundsException(errorClass = "INVALID_ARRAY_INDEX_IN_ELEMENT_AT",
      messageParameters = Array(index.toString, numElements.toString, SQLConf.ANSI_ENABLED.key))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: FAILED_EXECUTE_UDF
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 17:40
Affects Version/s: 3.4.0

Add at least one test for the error class *FAILED_EXECUTE_UDF* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def failedExecuteUserDefinedFunctionError(funcCls: String, inputTypes: String,
      outputType: String, e: Throwable): Throwable = {
    new SparkException(errorClass = "FAILED_EXECUTE_UDF",
      messageParameters = Array(funcCls, inputTypes, outputType), e)
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: CANNOT_CHANGE_DECIMAL_PRECISION
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 16:45
Affects Version/s: 3.4.0

Add at least one test for the error class *CANNOT_CHANGE_DECIMAL_PRECISION* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def cannotChangeDecimalPrecisionError(
      value: Decimal, decimalPrecision: Int, decimalScale: Int): ArithmeticException = {
    new SparkArithmeticException(errorClass = "CANNOT_CHANGE_DECIMAL_PRECISION",
      messageParameters = Array(value.toDebugString,
        decimalPrecision.toString, decimalScale.toString, SQLConf.ANSI_ENABLED.key))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: PySpark DeprecationWarning: distutils Version classes are deprecated
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 25/Mar/22 13:17
Affects Version/s: 3.2.1

When executing spark.read.csv(f'\{gcs_bucket}/\{data_file}', inferSchema=True, header=True) I'm getting the following warning:
{noformat}
.../lib/python3.8/site-packages/pyspark/sql/pandas/conversion.py:62: in toPandas
    require_minimum_pandas_version()
.../lib/python3.8/site-packages/pyspark/sql/pandas/utils.py:35: in require_minimum_pandas_version
    if LooseVersion(pandas.__version__) < LooseVersion(minimum_pandas_version):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <[AttributeError("'LooseVersion' object has no attribute 'vstring'") raised in repr()] LooseVersion object at 0x7f2319fc0f70>, vstring = '1.4.1'

    def __init__ (self, vstring=None):
>       warnings.warn(
            "distutils Version classes are deprecated. "
            "Use packaging.version instead.",
            DeprecationWarning,
            stacklevel=2,
        )
E       DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

.../lib/python3.8/site-packages/setuptools/_distutils/version.py:53: DeprecationWarning
{noformat}
Bug: Test the error class: INVALID_FRACTION_OF_SECOND
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 18:05
Affects Version/s: 3.4.0

Add a test for the error class *INVALID_FRACTION_OF_SECOND* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def invalidFractionOfSecondError(): DateTimeException = {
    new SparkDateTimeException(errorClass = "INVALID_FRACTION_OF_SECOND",
      Array(SQLConf.ANSI_ENABLED.key))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: DIVIDE_BY_ZERO
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 16:58
Affects Version/s: 3.4.0

Add at least one test for the error class *DIVIDE_BY_ZERO* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def divideByZeroError(): ArithmeticException = {
    new SparkArithmeticException(
      errorClass = "DIVIDE_BY_ZERO", messageParameters = Array(SQLConf.ANSI_ENABLED.key))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: CAST_CAUSES_OVERFLOW
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 16:49
Affects Version/s: 3.4.0

Add at least one test for the error class *CAST_CAUSES_OVERFLOW* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def castingCauseOverflowError(t: Any, dataType: DataType): ArithmeticException = {
    new SparkArithmeticException(errorClass = "CAST_CAUSES_OVERFLOW",
      messageParameters = Array(t.toString, dataType.catalogString, SQLConf.ANSI_ENABLED.key))
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Can't update spark.jars.packages on existing global/default context
Status: Open | Priority: Minor | Resolution: 
Created: 07/Mar/22 22:55
Affects Version/s: 3.2.1

Reproduction:

{code:python}
from pyspark.sql import SparkSession

# default session:
s = SparkSession.builder.getOrCreate()

# later on we want to update jars.packages, here's e.g. spark-hats
s = (SparkSession.builder
     .config("spark.jars.packages", "za.co.absa:spark-hats_2.12:0.2.2")
     .getOrCreate())

# line below returns None, the config was not propagated:
s._sc._conf.get("spark.jars.packages")
{code}

Stopping the context doesn't help, in fact it's even more confusing, because the configuration is updated, but doesn't have an effect:

{code:python}
from pyspark.sql import SparkSession

# default session:
s = SparkSession.builder.getOrCreate()

s.stop()

s = (SparkSession.builder
     .config("spark.jars.packages", "za.co.absa:spark-hats_2.12:0.2.2")
     .getOrCreate())

# now this line returns 'za.co.absa:spark-hats_2.12:0.2.2', but the context
# doesn't download the jar/package, as it would if there was no global context
# thus the extra package is unusable. It's not downloaded, or added to the
# classpath.
s._sc._conf.get("spark.jars.packages")
{code}

One workaround is to stop the context AND kill the JVM gateway, which seems to be a kind of hard reset:

{code:python}
from pyspark import SparkContext
from pyspark.sql import SparkSession

# default session:
s = SparkSession.builder.getOrCreate()

# Hard reset:
s.stop()
s._sc._gateway.shutdown()
s._sc._gateway.proc.stdin.close()
SparkContext._gateway = None
SparkContext._jvm = None

s = (SparkSession.builder
     .config("spark.jars.packages", "za.co.absa:spark-hats_2.12:0.2.2")
     .getOrCreate())

# Now we are guaranteed there's a new spark session, and packages
# are downloaded, added to the classpath etc.
{code}
Bug: Test the error class: CANNOT_PARSE_DECIMAL
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 16:47
Affects Version/s: 3.4.0

Add at least one test for the error class *CANNOT_PARSE_DECIMAL* to QueryExecutionErrorsSuite. The test should cover the exception throw in QueryExecutionErrors:

{code:scala}
  def cannotParseDecimalError(): Throwable = {
    new SparkIllegalStateException(errorClass = "CANNOT_PARSE_DECIMAL",
      messageParameters = Array.empty)
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class
Bug: Test the error class: DUPLICATE_KEY
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/Mar/22 17:00
Affects Version/s: 3.4.0

Add at least one test for the error class *DUPLICATE_KEY* to QueryParsingErrorsSuite. The test should cover the exception throw in QueryParsingErrors:

{code:scala}
  def duplicateKeysError(key: String, ctx: ParserRuleContext): Throwable = {
    // Found duplicate keys '$key'
    new ParseException(errorClass = "DUPLICATE_KEY", messageParameters = Array(key), ctx)
  }
{code}

For example, here is a test for the error class *UNSUPPORTED_FEATURE*: https://github.com/apache/spark/blob/34e3029a43d2a8241f70f2343be8285cb7f231b9/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala#L151-L170

+The test must have a check of:+
# the entire error message
# sqlState if it is defined in the error-classes.json file
# the error class