av9ash/GitBugs huggingface dataset:
Bug: Fix compilation warnings related to `auxiliary class TransportCipherUtil in TransportCipher.java should not be accessed from outside its own source file`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/Oct/24 17:05
Affects Version/s: 4.0.0

run 

build/mvn clean install -pl common/network-common

 

we can see

 
{code:java}
[WARNING] [Warn] /Users/yangjie01/SourceCode/git/spark-maven/common/network-common/src/main/java/org/apache/spark/network/crypto/CtrTransportCipher.java:73:11:  
auxiliary class TransportCipherUtil in /Users/yangjie01/SourceCode/git/spark-maven/common/network-common/src/main/java/org/apache/spark/network/crypto/TransportCipher.java should not be accessed from outside its own source file
[WARNING] [Warn] /Users/yangjie01/SourceCode/git/spark-maven/common/network-common/src/main/java/org/apache/spark/network/crypto/GcmTransportCipher.java:63:15:
auxiliary class TransportCipherUtil in /Users/yangjie01/SourceCode/git/spark-maven/common/network-common/src/main/java/org/apache/spark/network/crypto/TransportCipher.java should not be accessed from outside its own source file {code}
Bug: Utils.getCallSite() should also contain python stack traces
Status: Open | Priority: Major | Resolution: 
Created: 04/Oct/24 17:35
Affects Version/s: 3.5.3

Right now when using the pyspark the stages description looks like this:
{code:java}
save at NativeMethodAccessorImpl.java:0+detailsorg.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base/java.lang.reflect.Method.invoke(Method.java:566)
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
py4j.Gateway.invoke(Gateway.java:282)
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
py4j.commands.CallCommand.execute(CallCommand.java:79)
py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
py4j.ClientServerConnection.run(ClientServerConnection.java:106)
java.base/java.lang.Thread.run(Thread.java:829) {code}
Bug: Replace multiprocessing.ThreadPool in SparkConnect python code
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Oct/24 19:48
Affects Version/s: 3.5.4

multiprocessing.ThreadPool requires /dev/shm to be available, so the pyspark connect client doesn't work in environments where it's unavailable. We should replace it with concurrent.futures.ThreadPoolExecutor.
Bug: Extend Spark XSD parser into PySpark
Status: Open | Priority: Major | Resolution: 
Created: 03/Oct/24 10:14
Affects Version/s: 3.5.3

While PySpark does support XML parsing, it lacks a native XSD parser. Instead, you have to access the native Scala method {{XSDToSchema}} via the Spark Context.

This does work, but only if you are able to access the SparkContext object.  This is going to be a problem for users on Databricks, where Shared Clusters are required for many important features such as row-level security, since the Spark Context is not available from PySpark in this environment.

So I would like to request that the Apache Spark project implements a PySpark wrapper for the XSDToSchema Scala method. This should be a fairly quick and easy fix, as it is functionally the same as the existing PySpark function {{from_ddl_schema}}

This is implemented in PySpark already, [https://github.com/apache/spark/blob/master/python/pyspark/sql/types.py:1903:]

 
{code:java}
def from_ddl_schema(type_str: str) -> DataType:
            return _parse_datatype_json_string(
                cast(JVMView, sc._jvm)
                .org.apache.spark.sql.types.StructType.fromDDL(type_str)
                .json()
            ) {code}
And so an equivalent function for XSD might be as simple as:
{code:java}
def from_xsd_schema(type_str: str) -> DataType:
           return _parse_datatype_json_string(
                cast(JVMView, sc._jvm)
                .org.apache.spark.sql.execution.datasources.xml
                .XSDToSchema.read(type_str)
                .json()
            ) {code}
Bug: Pyspark JSON reader incorrectly considers a string of digits a timestamp and fails
Status: Open | Priority: Major | Resolution: 
Created: 02/Oct/24 18:52
Affects Version/s: 3.5.0

With pyspark 3.5.0 the reading the following JSON will fail:
{code:python}
from pyspark.sql import SparkSessionspark = SparkSession.builder.appName("timestamp_test").getOrCreate()
data = spark.sparkContext.parallelize(['{"field" : "23456"}'])
df = (
    spark.read.option("inferTimestamp", True)
    # .option("timestampFormat", "yyyy-MM-dd'T'HH:mm:ss[.SSS][XXX]")
    .json(path=data)
)
df.printSchema()
df.collect()
{code}
The printSchema command shows that the field is parsed as a timestamp, causing the following error:
{code:java}
File ~/miniforge3/envs/facets/lib/python3.11/site-packages/pyspark/sql/types.py:282, in TimestampType.fromInternal(self, ts)
    279 def fromInternal(self, ts: int) -> datetime.datetime:
    280     if ts is not None:
    281         # using int to avoid precision loss in float
--> 282         return datetime.datetime.fromtimestamp(ts // 1000000).replace(microsecond=ts % 1000000)

ValueError: year 23455 is out of range
{code}
If we uncomment the timestampFormat option, the command succeeds.

I believe there are two issues:
 # that a string of digits with length > 4 is inferred to be a timestamp
 # that setting timestampFormat to the default according to [the documentation|https://spark.apache.org/docs/3.5.0/sql-data-sources-json.html] fixes the problem such that the documented default is not the actual default.

This might be related to SPARK-45424.
Bug: PySpark requiring vulnerable Apache Zookeeper version 3.6.3
Status: Open | Priority: Major | Resolution: 
Created: 01/Oct/24 17:10
Affects Version/s: 3.5.3

When installing pyspark<4,

The package includes the /deps/jars/zookeeper-3.6.3.jar

This is being flagged as a high severity vulnerability in applications require pyspark 3.X as a dependency.

[CVE-2023-44981|https://zookeeper.apache.org/security.html#CVE-2023-44981]

This is despite the [pom.xml specifying zookeeper version 3.9.2|https://github.com/apache/spark/blob/3093ad68d2a3c6bab9c1605381d27e700766be22/pom.xml#L130]. It seems there is a bug with how this zookeeper dependency is being resolved.
Bug: File data sources don't respect provided user schema nullability for all APIs
Status: Open | Priority: Minor | Resolution: 
Created: 07/Oct/24 13:03
Affects Version/s: 4.0.0

DataFrameReader has 3 APIs for JSON reading

- json(DataSet[String])
- json(Rdd)
- json(filePath)

First two APIs respects provided user schema nullability when spark flag `spark.sql.legacy.respectNullabilityInTextDatasetConversion` is set to true, but third one does not respect and provided schema nullability is always overriden to true.

Some users just want to have a validation of data and to get exception when some field is nullable.
Bug: SALSA graph ranking algoritm upstreaming
Status: Open | Priority: Minor | Resolution: 
Created: 25/Oct/24 14:12
Affects Version/s: 3.1.2

Greetings,

A while back in 2021 as part of an MRes project I wrote several variations of the SALSA graph ranking algorithm for Apache Spark. I would like to know if there is any interest in upstreaming one of these versions in Apache Spark. I wrote some code to benchmark the performance and scalability of these variations. The code is available here:
[https://github.com/CajuM/DA/blob/salsa2/src/main/scala/PageRankBench/App.scala]

The benchmark is available here:
{{https://www.overleaf.com/read/yvhtkfqpcsbn#93e386}}
Bug: [k8s] Prefer to use Hadoop configMap when specified
Status: Open | Priority: Minor | Resolution: 
Created: 12/Oct/24 08:06
Affects Version/s: 3.5.1

In cloud vendor EMR environments, 'HADOOP_CONF_DIR' is often already configured in ENV. In this case, we want to preferentially use the configmap specified through the property spark.kubernetes.hadoop.configMapName.
Bug: Remove Hardcoded OnOutOfMemoryError Setting in Executor Startup Script
Status: Open | Priority: Minor | Resolution: 
Created: 31/Oct/24 06:12
Affects Version/s: 3.3.1

Currently, the Executor startup script hardcodes the addition of {{{}-XX:OnOutOfMemoryError='kill %p'{}}}, which causes the process to be killed when the Executor encounters an OOM (Out Of Memory) error.
{code:java}
// code in YarnSparkHadoopUtil
private[yarn] def addOutOfMemoryErrorArgument(javaOpts: ListBuffer[String]): Unit = {
  if (!javaOpts.exists(_.contains("-XX:OnOutOfMemoryError"))) {
    if (Utils.isWindows) {
      javaOpts += escapeForShell("-XX:OnOutOfMemoryError=taskkill /F /PID %%%%p")
    } else {
      javaOpts += "-XX:OnOutOfMemoryError='kill %p'"
    }
  }
}{code}
As a result, the YarnAllocator receives an exit code of 143 and is unable to accurately determine the reason for the Executor's termination based on this exit code. Moreover, the CoarseGrainedExecutorBackend cannot guarantee that StatusUpdate messages are sent to the Driver before the process is killed.
Could we remove this setting, since users can set it via the {{spark.executor.extraJavaOptions}} parameter if necessary?

Executor log:
!image-2024-10-31-14-14-06-723.png!
 
Driver log:
!image-2024-10-31-14-17-51-349.png!
Bug: UnresolvedException thrown instead of INVALID_LAMBDA_FUNCTION_CALL.NON_HIGHER_ORDER_FUNCTION
Status: Resolved | Priority: Minor | Resolution: Later
Created: 25/Oct/24 13:24
Affects Version/s: 4.0.0

Lambdas in non-higher order functions throw 
{code:java}
INVALID_LAMBDA_FUNCTION_CALL.NON_HIGHER_ORDER_FUNCTION{code}
, however this is only thrown from other functions, and not for all cases:
{code:java}
select if(x -> x = 2, 10, 20){code}
 throws 
{code:java}
INVALID_LAMBDA_FUNCTION_CALL.NON_HIGHER_ORDER_FUNCTION{code}
, but
{code:java}
select if(not x -> x = 2, 10, 20) {code}
throws 
{code:java}
UnresolvedException{code}
The user error in the latter case needs improvement
Bug: Intended way of using log4j plugin JARs
Status: Open | Priority: Minor | Resolution: 
Created: 22/Oct/24 13:53
Affects Version/s: 3.5.3

I want to use a log4j plugin, which would append the logs of Master JVM using classes from some 3rd party JARs:
[https://logging.apache.org/log4j/2.x/manual/plugins.html|https://logging.apache.org/log4j/2.x/manual/plugins.html#plugin-discovery]

The Spark configuration documentation says that I should configure the log4j via conf/log4j2.properties.
[https://spark.apache.org/docs/latest/configuration.html#configuring-logging]

This would require me to add my JARs on the class-path of Master, but it is not customizable and always is $SPARK_HOME/jars:

[https://github.com/apache/spark/blob/master/bin/spark-class#L71]

Is it intended to add such JARs to the $SPARK_HOME/jars?
Bug: Include SPARK_LOG_SCHEMA in the context of repl shell
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 18/Oct/24 23:07
Affects Version/s: 4.0.0

*Before the Change:*

Users needed to import {{LOG_SCHEMA}} to read structured logs as a JSON data source:
import org.apache.spark.util.LogUtils.LOG_SCHEMA

val logDf = spark.read.schema(LOG_SCHEMA).json("path/to/logs")
 
*After the Change:*
 * *No Import Needed in REPL Shells:* You can now use {{SPARK_LOG_SCHEMA}} directly in REPL environments like {{spark-shell}} and {{pyspark}} without importing it.
 * *Renamed for Clarity:* {{LOG_SCHEMA}} has been renamed to {{SPARK_LOG_SCHEMA}} to make its purpose more clear.

Now, you can read structured logs without the import:
val logDf = spark.read.schema(SPARK_LOG_SCHEMA).json("path/to/logs")
Bug: Show proper error message in memory profiler when it's not installed
Status: Resolved | Priority: Minor | Resolution: Invalid
Created: 20/Oct/24 06:58
Affects Version/s: 4.0.0

Running

{code}
from pyspark.sql.functions import pandas_udf
df = spark.range(10)

@pandas_udf("long")
def add1(x):
  return x + 1

spark.conf.set("spark.sql.pyspark.udf.profiler", "memory")

added = df.select(add1("id"))
added.show()
spark.profile.show(type="memory")
{code}

shows an error message like:

{code}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../python/pyspark/sql/profiler.py", line 320, in show
    self.profiler_collector.show_memory_profiles(id)
  File "/.../python/pyspark/sql/profiler.py", line 145, in show_memory_profiles
    show(id)
  File "/.../python/pyspark/sql/profiler.py", line 139, in show
    MemoryProfiler._show_results(cm)
  File "/.../python/pyspark/profiler.py", line 497, in _show_results
    tmp = template.format(lineno, total_mem, inc, occurrences, all_lines[lineno - 1])
{code}
Bug: Remove `javax.ws-rs-api` dependency
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 14/Oct/24 21:03
Affects Version/s: 4.0.0

Removing `javax.ws.rs-api` from root POM, as we used to have a few classes, ex: SecurityFilter.scala...etc rely on it, but overtime those classes either being removed / refactored. 

Hence we no longer could justify the import of `javax.ws.rs-api` package, and we should remove it.
Bug: Jakarta namespace migration
Status: Open | Priority: Minor | Resolution: 
Created: 14/Oct/24 20:43
Affects Version/s: 4.0.0

This is an umbrella ticket inspired by the Jetty upgrade tickets: SPARK-45522 and SPARK-47118.

 

Throughout the Jetty upgrade journey I realised there are a few `javax` dependencies exist on the codebase, hence this is the umbrella ticket to:
 # Investigate on each existing `javax` package on root POM and make sure the given `javax` is still relevant, in the context of Spark 4.0
 # Make the decision on either migrate it to `Jakarta` namespace, or remove it altogether, if it's not longer required. 

 

Also I hope by doing such clean up will help to pave a more clear and direct path for Jersey and Jetty upgrades down the road.
Bug: spark planing is too slow when LocalRelation has a lot of data
Status: Open | Priority: Trivial | Resolution: 
Created: 14/Oct/24 04:28
Affects Version/s: 3.5.1

```scala

import spark.implicits._

val data = (0 until 10000000).toArray

 

 // spend more than 10s

val ds = spark.createDataset(data) 

// spend more than 5s

ds.selectExpr("value+2 as value").selectExpr("value+3 as value").selectExpr("value+4 as value").selectExpr("value+5 as value") 

```

 

This is caused by `LocalRelation`, because `mapExpressions` will go into the data and spend a lot of time to tranversal it. Any idea to fix this issue?
Bug: SPJ: Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 02/Aug/23 18:53
Affects Version/s: 3.4.0

Adding the following test case in KeyGroupedPartitionSuite demonstrates the problem.

 
{code:java}
test("test join key is the second partition key and a transform") {
  val items_partitions = Array(bucket(8, "id"), days("arrive_time"))
  createTable(items, items_schema, items_partitions)
  sql(s"INSERT INTO testcat.ns.$items VALUES " +
    s"(1, 'aa', 40.0, cast('2020-01-01' as timestamp)), " +
    s"(1, 'aa', 41.0, cast('2020-01-15' as timestamp)), " +
    s"(2, 'bb', 10.0, cast('2020-01-01' as timestamp)), " +
    s"(2, 'bb', 10.5, cast('2020-01-01' as timestamp)), " +
    s"(3, 'cc', 15.5, cast('2020-02-01' as timestamp))")

  val purchases_partitions = Array(bucket(8, "item_id"), days("time"))
  createTable(purchases, purchases_schema, purchases_partitions)
  sql(s"INSERT INTO testcat.ns.$purchases VALUES " +
    s"(1, 42.0, cast('2020-01-01' as timestamp)), " +
    s"(1, 44.0, cast('2020-01-15' as timestamp)), " +
    s"(1, 45.0, cast('2020-01-15' as timestamp)), " +
    s"(2, 11.0, cast('2020-01-01' as timestamp)), " +
    s"(3, 19.5, cast('2020-02-01' as timestamp))")

  withSQLConf(
    SQLConf.REQUIRE_ALL_CLUSTER_KEYS_FOR_CO_PARTITION.key -> "false",
    SQLConf.V2_BUCKETING_PUSH_PART_VALUES_ENABLED.key -> "true",
    SQLConf.V2_BUCKETING_PARTIALLY_CLUSTERED_DISTRIBUTION_ENABLED.key ->
      "true") {
    val df = sql("SELECT id, name, i.price as purchase_price, " +
      "p.item_id, p.price as sale_price " +
      s"FROM testcat.ns.$items i JOIN testcat.ns.$purchases p " +
      "ON i.arrive_time = p.time " +
      "ORDER BY id, purchase_price, p.item_id, sale_price")

    val shuffles = collectShuffles(df.queryExecution.executedPlan)
    assert(!shuffles.isEmpty, "should not perform SPJ as not all join keys are partition keys")
    checkAnswer(df,
      Seq(
        Row(1, "aa", 40.0, 1, 42.0),
        Row(1, "aa", 40.0, 2, 11.0),
        Row(1, "aa", 41.0, 1, 44.0),
        Row(1, "aa", 41.0, 1, 45.0),
        Row(2, "bb", 10.0, 1, 42.0),
        Row(2, "bb", 10.0, 2, 11.0),
        Row(2, "bb", 10.5, 1, 42.0),
        Row(2, "bb", 10.5, 2, 11.0),
        Row(3, "cc", 15.5, 3, 19.5)
      )
    )
  }
}{code}
 

Note: this tests has setup the datasourceV2 to return multiple splits for same partition.

In this case, SPJ is not triggered (because join key does not match partition key), but the following code in DSV2Scan:

[https://github.com/apache/spark/blob/v3.4.1/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/BatchScanExec.scala#L194]

intended to fill the empty partition for 'pushdown-vallue' will still iterate through non-grouped partition and lookup from grouped partition to fill the map, resulting in some duplicate input data fed into the join.
Bug: Failure in testing `SparkSessionE2ESuite` using Maven
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 12/Aug/23 13:29
Affects Version/s: 3.5.0

[https://github.com/apache/spark/actions/runs/5832898984/job/15819181762]

 

The following failures exist in the daily Maven tests, we should fix them before Apache Spark 3.5.0 release:
{code:java}
SparkSessionE2ESuite:
4638- interrupt all - background queries, foreground interrupt *** FAILED ***
4639  The code passed to eventually never returned normally. Attempted 30 times over 20.092924822 seconds. Last failure message: Some("unexpected failure in q1: org.apache.spark.SparkException: org/apache/spark/sql/connect/client/SparkResult") was not empty Error not empty: Some(unexpected failure in q1: org.apache.spark.SparkException: org/apache/spark/sql/connect/client/SparkResult). (SparkSessionE2ESuite.scala:71)
4640- interrupt all - foreground queries, background interrupt *** FAILED ***
4641  "org/apache/spark/sql/connect/client/SparkResult" did not contain "OPERATION_CANCELED" Unexpected exception: org.apache.spark.SparkException: org/apache/spark/sql/connect/client/SparkResult (SparkSessionE2ESuite.scala:105)
4642- interrupt tag *** FAILED ***
4643  The code passed to eventually never returned normally. Attempted 30 times over 20.069445587 seconds. Last failure message: ListBuffer() had length 0 instead of expected length 2 Interrupted operations: ListBuffer().. (SparkSessionE2ESuite.scala:199)
4644- interrupt operation *** FAILED ***
4645  org.apache.spark.SparkException: org/apache/spark/sql/connect/client/SparkResult
4646  at org.apache.spark.sql.connect.client.GrpcExceptionConverter$.toThrowable(GrpcExceptionConverter.scala:89)
4647  at org.apache.spark.sql.connect.client.GrpcExceptionConverter$.convert(GrpcExceptionConverter.scala:38)
4648  at org.apache.spark.sql.connect.client.GrpcExceptionConverter$$anon$1.hasNext(GrpcExceptionConverter.scala:46)
4649  at org.apache.spark.sql.connect.client.SparkResult.org$apache$spark$sql$connect$client$SparkResult$$processResponses(SparkResult.scala:83)
4650  at org.apache.spark.sql.connect.client.SparkResult.operationId(SparkResult.scala:174)
4651  at org.apache.spark.sql.SparkSessionE2ESuite.$anonfun$new$31(SparkSessionE2ESuite.scala:243)
4652  at org.apache.spark.sql.connect.client.util.RemoteSparkSession.$anonfun$test$1(RemoteSparkSession.scala:243)
4653  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
4654  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
4655  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
4656  ... {code}
Bug: Enable and fix test_parity_arrow_python_udf
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 18/Aug/23 17:55
Affects Version/s: 3.5.0

{{pyspark.sql.tests.connect.test_parity_arrow_python_udf}} is not listed in {{dev/sparktestsupport/modules.py}}, and it fails when running manually.

{code}
======================================================================
ERROR [0.072s]: test_register (pyspark.sql.tests.connect.test_parity_arrow_python_udf.ArrowPythonUDFParityTests)
----------------------------------------------------------------------
Traceback (most recent call last):
...
pyspark.errors.exceptions.base.PySparkRuntimeError: [SCHEMA_MISMATCH_FOR_PANDAS_UDF] Result vector from pandas_udf was not the required length: expected 1, got 38.
{code}
Bug: Unable to read from JDBC data sources when using custom schema containing varchar
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 02/Aug/23 16:33
Affects Version/s: 3.1.0

When querying the data from JDBC databases with custom schema containing varchar I got this error :
{code:java}
[23/07/14 06:12:19 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 1) ( executor 1): java.sql.SQLException: Unsupported type varchar(100) at org.apache.spark.sql.errors.QueryExecutionErrors$.unsupportedJdbcTypeError(QueryExecutionErrors.scala:818) 23/07/14 06:12:21 INFO TaskSetManager: Lost task 0.1 in stage 1.0 (TID 2) on , executor 0: java.sql.SQLException (Unsupported type varchar(100)){code}
Code example: 
{code:java}
CUSTOM_SCHEMA="ID Integer, NAME VARCHAR(100)"
df = spark.read.format("jdbc")
.option("url", "jdbc:oracle:thin:@0.0.0.0:1521:db")
.option("driver", "oracle.jdbc.OracleDriver")
.option("dbtable", "table")
.option("customSchema", CUSTOM_SCHEMA)
.option("user", "user")
.option("password", "password")
.load()
df.show(){code}
I tried to set {{spark.sql.legacy.charVarcharAsString = true}} to restore the behavior before Spark 3.1 but it doesn't help.
The issue occurs in version 3.1.0 and above. I believe that this issue is caused by https://issues.apache.org/jira/browse/SPARK-33480
Bug: Fix PERCENTILE_DISC behaviour
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 18/Aug/23 12:54
Affects Version/s: 3.3.0

Currently {{percentile_disc()}} returns incorrect results in some cases:

E.g.:
{code:java}
SELECT
  percentile_disc(0.0) WITHIN GROUP (ORDER BY a) as p0,
  percentile_disc(0.1) WITHIN GROUP (ORDER BY a) as p1,
  percentile_disc(0.2) WITHIN GROUP (ORDER BY a) as p2,
  percentile_disc(0.3) WITHIN GROUP (ORDER BY a) as p3,
  percentile_disc(0.4) WITHIN GROUP (ORDER BY a) as p4,
  percentile_disc(0.5) WITHIN GROUP (ORDER BY a) as p5,
  percentile_disc(0.6) WITHIN GROUP (ORDER BY a) as p6,
  percentile_disc(0.7) WITHIN GROUP (ORDER BY a) as p7,
  percentile_disc(0.8) WITHIN GROUP (ORDER BY a) as p8,
  percentile_disc(0.9) WITHIN GROUP (ORDER BY a) as p9,
  percentile_disc(1.0) WITHIN GROUP (ORDER BY a) as p10
FROM VALUES (0), (1), (2), (3), (4) AS v(a)
{code}
returns:
{code:java}
+---+---+---+---+---+---+---+---+---+---+---+
| p0| p1| p2| p3| p4| p5| p6| p7| p8| p9|p10|
+---+---+---+---+---+---+---+---+---+---+---+
|0.0|0.0|0.0|1.0|1.0|2.0|2.0|2.0|3.0|3.0|4.0|
+---+---+---+---+---+---+---+---+---+---+---+
{code}
but it should return:
{noformat}
+---+---+---+---+---+---+---+---+---+---+---+
| p0| p1| p2| p3| p4| p5| p6| p7| p8| p9|p10|
+---+---+---+---+---+---+---+---+---+---+---+
|0.0|0.0|0.0|1.0|1.0|2.0|2.0|3.0|3.0|4.0|4.0|
+---+---+---+---+---+---+---+---+---+---+---+
{noformat}