av9ash/GitBugs huggingface dataset:
Bug: Release restrictions on multi-layer runtime filter
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/Oct/23 07:54
Affects Version/s: 4.0.0

Before https://issues.apache.org/jira/browse/SPARK-41674, Spark only supports insert runtime filter for application side of shuffle join on single-layer. Considered it's not worth to insert more runtime filter if one side of the shuffle join already exists runtime filter, Spark restricts it.

After https://issues.apache.org/jira/browse/SPARK-41674, Spark supports insert runtime filter for one side of any shuffle join on multi-layer. But the restrictions on multi-layer runtime filter looks outdated.
Bug: Assign name to _LEGACY_ERROR_TEMP_21[59,60,61,62]
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 12:19
Affects Version/s: 3.5.0

Choose a proper name for the error class {*}_LEGACY_ERROR_TEMP_21[59,60,61,62]{*}defined in {*}core/src/main/resources/error/error-classes.json{*}. The name should be short but complete (look at the example in error-classes.json).

Add a test which triggers the error from user code if such test still doesn't exist. Check exception fields by using {*}checkError(){*}. The last function checks valuable error fields only, and avoids dependencies from error text message. In this way, tech editors can modify error format in error-classes.json, and don't worry of Spark's internal tests. Migrate other tests that might trigger the error onto checkError().

If you cannot reproduce the error from user space (using SQL query), replace the error by an internal error, see {*}SparkException.internalError(){*}.

Improve the error message format in error-classes.json if the current is not clear. Propose a solution to users how to avoid and fix such kind of errors.

Please, look at the PR below as examples:
 * [https://github.com/apache/spark/pull/38685]
 * [https://github.com/apache/spark/pull/38656]
 * [https://github.com/apache/spark/pull/38490]
Bug: Clean up the deprecated API usage related to `MapOps`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 02:24
Affects Version/s: 4.0.0

* method - in trait MapOps is deprecated (since 2.13.0)
 * method -- in trait MapOps is deprecated (since 2.13.0)
 * method + in trait MapOps is deprecated (since 2.13.0)

 
{code:java}
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/core/src/main/scala/org/apache/spark/deploy/worker/CommandUtils.scala:84:27: method + in trait MapOps is deprecated (since 2.13.0): Consider requiring an immutable Map or fall back to Map.concat.
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.deploy.worker.CommandUtils.buildLocalCommand.newEnvironment, origin=scala.collection.MapOps.+, version=2.13.0
[warn]       command.environment + ((libraryPathName, libraryPaths.mkString(File.pathSeparator)))
[warn]                           ^
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/core/src/main/scala/org/apache/spark/deploy/worker/CommandUtils.scala:91:22: method + in trait MapOps is deprecated (since 2.13.0): Consider requiring an immutable Map or fall back to Map.concat.
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.deploy.worker.CommandUtils.buildLocalCommand, origin=scala.collection.MapOps.+, version=2.13.0
[warn]       newEnvironment += (SecurityManager.ENV_AUTH_SECRET -> securityMgr.getSecretKey())
[warn]                      ^ {code}
Bug: Fix `method += in trait Growable is deprecated`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 02:33
Affects Version/s: 4.0.0

{code:java}
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:730:47: method += in trait Growable is deprecated (since 2.13.0): Use `++=` aka `addAll` instead of varargs `+=`; infix operations with an operand of multiple args will be deprecated
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment.$anonfun, origin=scala.collection.mutable.Growable.+=, version=2.13.0
[warn]         if (opt.clOption != null) { childArgs += (opt.clOption, opt.value) }
[warn]                                               ^ {code}
Bug: Introduce an implicit function for Scala Array to wrap into `immutable.ArraySeq`.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/Oct/23 18:45
Affects Version/s: 4.0.0

Currently, we need to use `immutable.ArraySeq.unsafeWrapArray(array)` to wrap an Array into an `immutable.ArraySeq`, which makes the code look bloated.
Bug: Assign name to _LEGACY_ERROR_TEMP_215[6,7,8]
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 20/Oct/23 02:54
Affects Version/s: 3.5.0

Choose a proper name for the error class *_LEGACY_ERROR_TEMP_2153* defined in {*}core/src/main/resources/error/error-classes.json{*}. The name should be short but complete (look at the example in error-classes.json).

Add a test which triggers the error from user code if such test still doesn't exist. Check exception fields by using {*}checkError(){*}. The last function checks valuable error fields only, and avoids dependencies from error text message. In this way, tech editors can modify error format in error-classes.json, and don't worry of Spark's internal tests. Migrate other tests that might trigger the error onto checkError().

If you cannot reproduce the error from user space (using SQL query), replace the error by an internal error, see {*}SparkException.internalError(){*}.

Improve the error message format in error-classes.json if the current is not clear. Propose a solution to users how to avoid and fix such kind of errors.

Please, look at the PR below as examples:
 * [https://github.com/apache/spark/pull/38685]
 * [https://github.com/apache/spark/pull/38656]
 * [https://github.com/apache/spark/pull/38490]
Bug: Catch IOException instead of EOFException alone for faulthandler
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/Oct/23 11:32
Affects Version/s: 4.0.0

{{spark.python.worker.faulthandler.enabled}} can describe the fatal exception such as segfault hanlder. Exceptions such as {{java.net.SocketException: Connection reset}} can happen because the worker unexpectedly die. We should better catch all IO exception there.
Bug: Introduce a mapper for parquet compression codecs
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 10/Oct/23 08:57
Affects Version/s: 4.0.0

Currently, Spark supported all the parquet compression codecs, but the parquet supported compression codecs and spark supported are not completely one-on-one due to Spark introduce a fake compression codecs none.

There are a lot of magic strings copy from parquet compression codecs. This issue lead to developers need to manually maintain its consistency. It is easy to make mistakes and reduce development efficiency.
Bug: Support registering Python data sources
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 22:05
Affects Version/s: 4.0.0

Support registering Python data sources.

Users can register a Python data source and later use reference it using its name.
{code:java}
class MyDataSource(DataSource):
    @classmethod
    def name(cls):
        return "my-data-source"

spark.dataSource.register(MyDataSource){code}
Users can then use the name of the data source as the format (will be supported in SPARK-45639)
{code:java}
spark.read.format("my-data-source").load(){code}
Bug: Remove unnecessary `.toArray[InternalRow]` in `SparkPlan#executeTake` function.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/Oct/23 08:49
Affects Version/s: 4.0.0

{code:java}
if (takeFromEnd) {
  while (buf.length < n && i < res.length) {
    val rows = decodeUnsafeRows(res(i)._2)
    if (n - buf.length >= res(i)._1) {
      buf.prependAll(rows.toArray[InternalRow])
    } else {
      val dropUntil = res(i)._1 - (n - buf.length)
      // Same as Iterator.drop but this only takes a long.
      var j: Long = 0L
      while (j < dropUntil) { rows.next(); j += 1L}
      buf.prependAll(rows.toArray[InternalRow])
    }
    i += 1
  }
} else {
  while (buf.length < n && i < res.length) {
    val rows = decodeUnsafeRows(res(i)._2)
    if (n - buf.length >= res(i)._1) {
      buf ++= rows.toArray[InternalRow]
    } else {
      buf ++= rows.take(n - buf.length).toArray[InternalRow]
    }
    i += 1
  }
} {code}
In the above code, the input parameters of `mutable.Buffer#prependAll` and `mutable.Growable#++=` functions are `IterableOnce`, and the type of rows is `Iterator[InternalRow]`, which inherits from `IterableOnce`, so there is no need to cast to an array of InternalRow anymore.
Bug: Fix `abstract type TypeA in type pattern Some[TypeA] is unchecked since it is eliminated by erasure`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 04:11
Affects Version/s: 4.0.0

{code:java}
[error] /Users/yangjie01/SourceCode/git/spark-mine-sbt/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala:105:19: abstract type ScalaInputType in type pattern Some[ScalaInputType] is unchecked since it is eliminated by erasure
[error] Applicable -Wconf / @nowarn filters for this fatal warning: msg=<part of the message>, cat=unchecked, site=org.apache.spark.sql.catalyst.CatalystTypeConverters.CatalystTypeConverter.toCatalyst
[error]         case opt: Some[ScalaInputType] => toCatalystImpl(opt.get)
[error]                   ^ {code}
Bug: Fix `the type test for pattern TypeA cannot be checked at runtime`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 04:06
Affects Version/s: 4.0.0

{code:java}
[error] /Users/yangjie01/SourceCode/git/spark-mine-sbt/core/src/main/scala/org/apache/spark/rdd/OrderedRDDFunctions.scala:100:21: the type test for pattern org.apache.spark.RangePartitioner[K,V] cannot be checked at runtime because it has type parameters eliminated by erasure
[error] Applicable -Wconf / @nowarn filters for this fatal warning: msg=<part of the message>, cat=unchecked, site=org.apache.spark.rdd.OrderedRDDFunctions.filterByRange.rddToFilter
[error]       case Some(rp: RangePartitioner[K, V]) =>
[error]  {code}
Bug: Fix `method any2stringadd in object Predef is deprecated`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 02:10
Affects Version/s: 4.0.0

{code:java}
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala:720:17: method any2stringadd in object Predef is deprecated (since 2.13.0): Implicit injection of + is deprecated. Convert to String to call +
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.sql.catalyst.expressions.BinaryExpression.nullSafeCodeGen.nullSafeEval, origin=scala.Predef.any2stringadd, version=2.13.0
[warn]         leftGen.code + ctx.nullSafeExec(left.nullable, leftGen.isNull) {
[warn]                 ^ {code}
Bug: Clean up the deprecated API usage related to `Iterator/Iterable/IterableOps`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 02:32
Affects Version/s: 4.0.0

* method seq in trait Iterator is deprecated (since 2.13.0)
 * method toIterable in trait Iterable is deprecated (since 2.13.7)
 * method seq in trait Iterable is deprecated (since 2.13.0)
 * method toTraversable in trait IterableOps is deprecated (since 2.13.0)

 
{code:java}
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/mllib/src/test/scala/org/apache/spark/ml/feature/NormalizerSuite.scala:79:55: method seq in trait Iterable is deprecated (since 2.13.0): Iterable.seq always returns the iterable itself
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.ml.feature.NormalizerSuite, origin=scala.collection.Iterable.seq, version=2.13.0
[warn]     val dataFrame: DataFrame = data.zip(l2Normalized).seq.toDF("features", "expected") {code}
Bug:   Clean up the deprecated API usage related to `StringContext/StringOps`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 02:26
Affects Version/s: 4.0.0

 
{code:java}
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala:258:30: method treatEscapes in object StringContext is deprecated (since 2.13.0): use processEscapes
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.sql.catalyst.expressions.codegen.Block.foldLiteralArgs, origin=scala.StringContext.treatEscapes, version=2.13.0
[warn]     buf.append(StringContext.treatEscapes(strings.next()))
[warn]                              ^
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala:270:32: method treatEscapes in object StringContext is deprecated (since 2.13.0): use processEscapes
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.sql.catalyst.expressions.codegen.Block.foldLiteralArgs, origin=scala.StringContext.treatEscapes, version=2.13.0
[warn]       buf.append(StringContext.treatEscapes(strings.next()))
[warn]   {code}
 

 
 * method checkLengths in class StringContext is deprecated (since 2.13.0)
 * method treatEscapes in object StringContext is deprecated (since 2.13.0)
 * method replaceAllLiterally in class StringOps is deprecated (since 2.13.2)
Bug:   Clean up the deprecated API usage related to `RightProjection/LeftProjection/Either`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 02:31
Affects Version/s: 4.0.0

* method get in class RightProjection is deprecated (since 2.13.0)
 * method get in class LeftProjection is deprecated (since 2.13.0)
 * method right in class Either is deprecated (since 2.13.0)

 
{code:java}
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/GroupBasedRowLevelOperationScanPlanning.scala:54:28: method get in class LeftProjection is deprecated (since 2.13.0): use `Either.swap.getOrElse` instead
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.sql.execution.datasources.v2.GroupBasedRowLevelOperationScanPlanning.apply, origin=scala.util.Either.LeftProjection.get, version=2.13.0
[warn]         pushedFilters.left.get.mkString(", ")
[warn]                            ^ {code}
Bug: Clean up type use of `BufferedIterator/CanBuildFrom/Traversable`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 02:29
Affects Version/s: 4.0.0

* type BufferedIterator in package scala is deprecated (since 2.13.0)
 * type CanBuildFrom in package generic is deprecated (since 2.13.0)
 * type Traversable in package scala is deprecated (since 2.13.0)

 
{code:java}
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/sql/core/src/main/scala/org/apache/spark/sql/execution/GroupedIterator.scala:67:12: type BufferedIterator in package scala is deprecated (since 2.13.0): Use scala.collection.BufferedIterator instead of scala.BufferedIterator
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.sql.execution.GroupedIterator.input, origin=scala.BufferedIterator, version=2.13.0
[warn]     input: BufferedIterator[InternalRow],
[warn]            ^ {code}
Bug: False positive when cheking for ambigious columns 
Status: Open | Priority: Major | Resolution: 
Created: 30/Oct/23 10:18
Affects Version/s: 3.4.0

I have following code, which I expect to work
{code:java}
from pyspark.sql import SparkSession

import pyspark.sql.functions as F session = SparkSession.Builder().getOrCreate() A = session.createDataFrame([(1,)], ['a'])
B = session.createDataFrame([(1,)], ['b'])
A.join(B).select(B.b) # works fine
C = A.join(A.join(B), on=F.lit(False), how='leftanti') # C has the same columns as A (columns, not only names)
C.join(B).select(B.b) #doesn't work, says B.b is ambigious,
{code}
{code:java}
Exception below:{code}
{code:java}
AnalysisException: Column b#11L are ambiguous. It's probably because you joined several Datasets together, and some of these Datasets are the same. This column points to one of the Datasets but Spark is unable to figure out which one. Please alias the Datasets with different names via `Dataset.as` before joining them, and specify the column using qualified name, e.g. `df.as("a").join(df.as("b"), $"a.id" > $"b.id")`. You can also set spark.sql.analyzer.failAmbiguousSelfJoin to false to disable this check.{code}
Bug: Introduce a mapper for orc compression codecs
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/Oct/23 09:18
Affects Version/s: 4.0.0

Currently, Spark supported all the orc compression codecs, but the orc supported compression codecs and spark supported are not completely one-on-one due to Spark introduce two compression codecs none and UNCOMPRESSED.

There are a lot of magic strings copy from orc compression codecs. This issue lead to developers need to manually maintain its consistency. It is easy to make mistakes and reduce development efficiency.
Bug: Enhancing clarity and usability of PySpark error messages
Status: Reopened | Priority: Major | Resolution: 
Created: 26/Oct/23 06:00
Affects Version/s: 4.0.0

It's essential for users to understand error messages clearly to troubleshoot issues efficiently. This umbrella ticket aims to gather feedback from the PySpark community regarding areas where error messages could be improved for clarity, accuracy, and user-friendliness.

Users are encouraged to:
 # Create sub-tasks to address specific error messages or categories of error messages that they believe can be enhanced.
 # Provide examples of the current error message and suggestions for improvement.
 # Describe the context or scenarios where the error is encountered to provide better insight into the issue.

By collaboratively identifying and refining ambiguous or misleading error messages, we hope to make PySpark a more user-friendly and accessible tool for all users.
Bug: Spark connect executor ignore jars in classpath
Status: Resolved | Priority: Major | Resolution: Invalid
Created: 17/Oct/23 11:21
Affects Version/s: 3.5.0

I am starting a spark connect server in k8s.

While trying to access S3 I'm getting the following error from the executor
{code:java}
Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3) (10.1.0.174 executor 1): java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2688)
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3431)
... {code}
The driver is able to access S3 without trouble.

I tried adding the hadoop-aws jar in multiple ways: --packcages, --jars, 
SPARK_EXTRA_CLASSPATH env and adding to the spark Jars folder in my dockerfile.
 
When looking in my executor pod, I'm seeing the classpath is set up properly and should have access to the jar located in multiple places.

 

Only by using the addArtifact API I can add the missing jar.
Bug: Improve error message for JVM-dependent attributes on Spark Connect.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Oct/23 06:02
Affects Version/s: 4.0.0

When trying to access to `sparkContext` on Spark Connect, `[NOT_IMPLEMENTED] sparkContext() is not implemented.` is raised, but it is not clear to describe why sparkContext is not available from Spark Connect.
Bug: Introduce a mapper for avro compression codecs
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 12:43
Affects Version/s: 4.0.0

Currently, Spark supported all the avro compression codecs, but the avro supported compression codecs and spark supported are not completely one-on-one due to Spark introduce the compression codecs UNCOMPRESSED.

There are a lot of magic strings copy from avro compression codecs. This issue lead to developers need to manually maintain its consistency. It is easy to make mistakes and reduce development efficiency.
Bug: Snapshots of some packages are not published any more
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Oct/23 15:52
Affects Version/s: 4.0.0

Snapshots of some packages are not been published anymore, e.g. spark-sql_2.13-4.0.0 has not been published since Sep, 13th: https://repository.apache.org/content/groups/snapshots/org/apache/spark/spark-sql_2.13/4.0.0-SNAPSHOT/

There have been some attempts to fix CI: SPARK-45535 SPARK-45536

Assumption is that memory consumption during build exceeds the available memory of the Github host.

The following could be attempted:
- enable manual trigger of the {{publish_snapshots.yml}} workflow
- enable some memory use logging to proof that exceeded memory is the root cause
- attempt to reduce memory footprint and see impact in above logging
- revert memory use logging
Bug: Clean up the deprecated API usage related to `Buffer`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 03:10
Affects Version/s: 4.0.0

* method append in trait Buffer is deprecated (since 2.13.0)
 * method prepend in trait Buffer is deprecated (since 2.13.0)
 * method trimEnd in trait Buffer is deprecated (since 2.13.4)
 * method trimStart in trait Buffer is deprecated (since 2.13.4)

{code:java}
[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/core/src/test/scala/org/apache/spark/deploy/IvyTestUtils.scala:319:18: method append in trait Buffer is deprecated (since 2.13.0): Use appendAll instead
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.deploy.IvyTestUtils.createLocalRepository, origin=scala.collection.mutable.Buffer.append, version=2.13.0
[warn]         allFiles.append(rFiles: _*)
[warn]                  ^ 

[warn] /Users/yangjie01/SourceCode/git/spark-mine-sbt/core/src/main/scala/org/apache/spark/util/SizeEstimator.scala:183:13: method trimEnd in trait Buffer is deprecated (since 2.13.4): use dropRightInPlace instead
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.util.SizeEstimator.SearchState.dequeue, origin=scala.collection.mutable.Buffer.trimEnd, version=2.13.4
[warn]       stack.trimEnd(1)
[warn]             ^{code}
Bug: Makes entire Binder build fails fast during setting up
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Oct/23 08:44
Affects Version/s: 3.5.0

Binder build is currently broken:

https://mybinder.org/v2/gh/apache/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb

Seems like we uploaded PySpark late into PyPI, and the installation steps just slightly ignored the failure.
Bug: Add clusterBy in DataFrame API
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Oct/23 21:20
Affects Version/s: 3.5.1

Add clusterBy to Dataframe API e.g. in python

DataframeWriterV1
```
df.write
  .format("delta")
  .clusterBy("clusteringColumn1", "clusteringColumn2")
  .save(...) or saveAsTable(...)
```

DataFrameWriterV2
```
df.writeTo(...).using("delta")
  .clusterBy("clusteringColumn1", "clusteringColumn2")
  .create() or replace() or createOrReplace()
```
Bug: Observe API error logging
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Oct/23 17:37
Affects Version/s: 4.0.0

We should tell user why it's not supported and what to do

[https://github.com/apache/spark/blob/536439244593d40bdab88e9d3657f2691d3d33f2/sql/core/src/main/scala/org/apache/spark/sql/Observation.scala#L76]