av9ash/GitBugs huggingface dataset:
Bug: Hive MetaException attempting to get partition metadata by filter from Hive
Status: Open | Priority: Major | Resolution: 
Created: 04/May/21 19:51
Affects Version/s: 3.0.1

Got an error while running a code through Airflow DAG.

Exception while running an ETL job on an External table created on Hive stored as parquet in S3 with AWS Glue as metastore. Here's the error message:

 

java.lang.RuntimeException: Caught Hive MetaException attempting to get partition metadata by filter from Hive. You can set the Spark configuration setting spark.sql.hive.manageFilesourcePartitions to false to work around this problem, however this will result in degraded performance. Please report a bug: https://issues.apache.org/jira/browse/SPARK |

 

Caused by: MetaException(message:Unknown exception occurred. (Service: AWSGlue; Status Code: 500; Error Code: InternalServiceException; Request ID: 73267997-1795-45a3-965f-8bb2a6b7b3ac))

 

Exact issue occurred while running on Databricks notebook as well. Screenshot attached for both cases.
Bug: Improve InferFiltersFromConstraints rule performance when parsing spark sql
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 12/May/21 03:52
Affects Version/s: 3.1.1

*InferFiltersFromConstraints* rule will generates too many constraints when there are many aliases in Project, . For example:

 
{code:java}

test("Expression explosion when analyze test") {
    RuleExecutor.resetMetrics()
    Seq((1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
      .toDF("a", "b", "c", "d", "e", "f", "g", "h", "i", "j",
        "k", "l", "m", "n")
      .write.saveAsTable("test")
    val df = spark.table("test")
    val df2 = df.filter("a+b+c+d+e+f+g+h+i+j+k+l+m+n > 100")
    val df3 = df2.select('a as 'a1, 'b as 'b1,
      'c as 'c1, 'd as 'd1, 'e as 'e1, 'f as 'f1,
      'g as 'g1, 'h as 'h1, 'i as 'i1, 'j as 'j1,
      'k as 'k1, 'l as 'l1, 'm as 'm1, 'n as 'n1)
    val df4 = df3.join(df2, df3("a1") === df2("a"))
    df4.explain(true)
    logWarning(RuleExecutor.dumpTimeSpent())
  }

{code}


So we need to optimize *InferFiltersFromConstraints* rule to improve performance.
Bug: Use MethodUtils for method looking up in Invoke and StaticInvoke
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/May/21 05:53
Affects Version/s: 3.2.0

Currently we wrote our logic in Invoke and StaticInvoke expressions for looking up methods. It is tricky to consider all the cases and there is already existing utility package for this purpose. We should reuse the utility package.
Bug: Window group-key and pandas inconsistent
Status: Open | Priority: Major | Resolution: 
Created: 11/May/21 06:55
Affects Version/s: 3.0.1

Not completely sure whether this is a bug or a configuration/usage issue:

We are seeing inconsistent timezone-treatment when using a windowed group by aggregation in combiation with a pandas udf.

A minimal example:

 
{code:java}
// code placeholder

def a_udf(group_key, pdf: pd.DataFrame) -> pd.DataFrame:
    w_start = group_key[0]["start"]
    w_end = group_key[0]["end"]

    print(f"Pandas   : {pdf['window_start'].iloc[0]} to {pdf['window_end'].iloc[0]}")
    print(f"Group key: {w_start} to {w_end}")
    print(f"Data     : {pdf['time'].min()} to {pdf['time'].max()}")

    assert (pdf["time"] >= w_start).all()
    assert (pdf["time"] < w_end).all()        

    # some result

    return pd.DataFrame.from_records([{"result": 1}])


df = spark.createDataFrame([(datetime.datetime(2020, 1, 1, 12, 30, 0),)], schema=["time"])

w = window("time", "60 minutes")
df.withColumn("window_start", w.start)\
  .withColumn("window_end", w.end)\
  .groupby(w).applyInPandas(a_udf, schema="result int")\
  .show()
{code}
 

Produces:
{code:java}
Pandas   : 2020-01-01 12:00:00 to 2020-01-01 13:00:00
Group key: 2020-01-01 11:00:00 to 2020-01-01 12:00:00
Data     : 2020-01-01 12:30:00 to 2020-01-01 12:30:00

{code}
And the assertions fail. It seems the group-key goes through some timezone- (and dst?) conversion that ends up being one hour off.
 This is without any specific timezone configuration and with CEST as the local timezone.

Is this working as expected?

Setting
{code:java}
"spark.sql.session.timeZone": "UTC"
"spark.driver.extraJavaOptions": "-Duser.timezone=UTC"
"spark.executor.extraJavaOptions": "-Duser.timezone=UTC"
{code}
seems to be workaround.
 Using "Europe/Berlin" for all three timezone settings however reproduces the inconsistent behaviour.

I would assume though, that running this in a non-UTC timezone should generally be possible?
Bug: Attributes become unknown in RepartitionByExpression after aliased
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/May/21 03:47
Affects Version/s: 2.4.8


{code:java}
explain extended select a b from values(1) t(a) distribute by a;
== Parsed Logical Plan ==
'RepartitionByExpression ['a]
+- 'Project ['a AS b#42]
   +- 'SubqueryAlias t
      +- 'UnresolvedInlineTable [a], [List(1)]

== Analyzed Logical Plan ==
org.apache.spark.sql.AnalysisException: cannot resolve 'a' given input columns: [b]; line 1 pos 62;
'RepartitionByExpression ['a]
+- Project [a#48 AS b#42]
   +- SubqueryAlias t
      +- LocalRelation [a#48]
{code}
Bug: StaticInvoke should find the method without exact argument classes match
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/May/21 20:19
Affects Version/s: 2.4.7

Unlike Invoke, StaticInvoke only tries to get the method with exact argument classes match. If the calling method's parameter types are not exactly matched with the argument classes, StaticInvoke cannot find the method.

StaticInvoke should be able to find the method under the cases too.
Bug: Pyspark - Using importlib + filter + named function + take causes pyspark to restart continuously until machine runs out of memory
Status: Open | Priority: Major | Resolution: 
Created: 07/May/21 13:52
Affects Version/s: 3.0.0

Repo to reproduce issue

[https://github.com/CanaryWharf/pyspark-mem-importlib-bug-reproduction]

 

Expected behavour:

Program runs and exits cleanly

 

Actual behaviour:

Program runs forever, eating up all the memory on the machine

 

Steps to reproduce:

```

pip install -r requirements.txt

python run.py

``` 

The problem only occurs if you run the code via `importlib`. The problem does not occur running `sparky.py` directly.

Furthermore, the problem occurs if you replace filter with map or flatMap (anything that takes in a lambda function).

The problem only occurs if you call a named function (i.e., when you use `def func`).

So these break:

```

def func(stuff):

    return True

 

dataset.filter(func)

 ```

 

```

def func(stuff):

    return True

 

dataset.filter(lambda s: func(s))

 ```

 

The problem does *NOT* occur if you do this:

```

dataset.filter(lambda x: True)

```

```

dataset.filter(lambda x: x == 'stuff')

 ```
Bug: Tune shuffle requests frequency dynamically in the case of Netty OOM
Status: Open | Priority: Major | Resolution: 
Created: 06/May/21 15:21
Affects Version/s: 3.2.0

In [https://github.com/apache/spark/pull/32287,] the PR proposes to use a flag to indicate the Netty OOM status and defer the fetch requests when OOM happens. However, it doesn't change the request frequency after OOM happens. Thus, it's possible to hit the OOM again later if the deferred requests are still too much to fetch concurrently. Therefore, tuning the fetch frequency might be a good idea to solve the issue. Please see the detailed discussion at https://github.com/apache/spark/pull/32287#discussion_r625287419
Bug: Use 'SPARK_LOG_URL_' as env prefix for getting driver log urls by default
Status: In Progress | Priority: Major | Resolution: 
Created: 06/May/21 14:03
Affects Version/s: 3.1.1

Currently, spark on kubernetes can't show logs url on ui. To check history logs, we usually collect pod logs to third-party logging services, which can be accessed by urls. To show log urls,  we can set env prefixed with 'SPARK_LOG_URL_' for executors. But for driver, there is no way to show log urls by set env. 

 

I will create a new pr that use 'SPARK_LOG_URL_' as env prefix for getting driver log urls by default.
Bug: array transform should respect alias in expression
Status: Resolved | Priority: Major | Resolution: Not A Bug
Created: 04/May/21 09:46
Affects Version/s: 3.1.0

When an array is transformed using the transform function in combination with an alias, I get an unexpected column name.
{code:scala}
// code placeholder
val data = Seq((Array(1, 2, 3)))
val df = sc.parallelize(data).toDF("a")

val dfRes = df.select(
  transform(
    $"a",
    (col: Column) => struct(col).alias("struct_alias")
  ).alias("a")
)

dfRes.printSchema
// root
//  |-- a: array (nullable = true)
//  |    |-- element: struct (containsNull = false)
//  |    |    |-- x: integer (nullable = false)
{code}

I expected the inner element to have the name 'struct_alias'.
Bug: Migrate to transformWithPruning for top-level rules under catalyst/optimizer
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 03/May/21 05:57
Affects Version/s: 3.1.0

**List of target rules:
 * DecorrelateInnerQuery
 * LimitPushDownThroughWindow
 * OptimizeCsvJsonExprs
 * PropagateEmptyRelation
 * PullOutGroupingExpressions
 * PushLeftSemiLeftAntiThroughJoin
 * ReplaceExceptWithFilter
 * RewriteDistinctAggregates
 * SimplifyConditionalsInPredicate
 * UnwrapCastInBinaryComparison

**
Bug: InMemoryFileIndex load all files into memroy
Status: Open | Priority: Minor | Resolution: 
Created: 19/May/21 07:43
Affects Version/s: 2.3.1

When InMemoryFileIndex is initialated, it'll load all files of ${rootPath}. If there're thousands of files, it could lead to OOM.

 

https://github.com/apache/spark/blob/0b3758e8cdb3eaa9d55ce3b41ecad5fa01567343/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala#L66
Bug: job group and description do not apply on broadcasts
Status: Open | Priority: Minor | Resolution: 
Created: 25/May/21 06:40
Affects Version/s: 3.0.0

Given the following code:
{code:java}
SparkContext context = new SparkContext("local", "test"); 
SparkSession session = new SparkSession(context); 

List<String> strings = Lists.newArrayList("a", "b", "c"); 
List<String> otherString = Lists.newArrayList( "b", "c", "d"); 

Dataset<Row> broadcastedDf = session.createDataset(strings, Encoders.STRING()).toDF(); 
Dataset<Row> dataframe = session.createDataset(otherString, Encoders.STRING()).toDF(); 

context.setJobGroup("my group", "my job", false); 
dataframe.join(broadcast(broadcastedDf), "value").count();

{code}
Job group and description do not apply on broadcasted dataframe. 
With spark 2.x, broadcast creation is given the same job description as the query itself. 
This seems to be broken with spark 3.x

See attached images
 !spark3-image.png!  !spark2-image.png!
Bug: Upgrade SBT to 1.5.2
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/May/21 22:52
Affects Version/s: 3.2.0

This will bring the following bug fix for Scala 2.13.4.
{code}
Fixes ConcurrentModificationException while compiling Scala 2.13.4 and Java sources zinc
{code}
Bug: Job submission in high performance computing environments
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 05/May/21 06:56
Affects Version/s: 3.1.2

Spark is often used in high performance computing environments. The default launcher does not directly support schedulers used such as [slurm|https://slurm.schedmd.com/overview.html] and [pbs|https://openpbs.org/].  It would be good to support these directly. The repositories [https://github.com/ekasitk/spark-on-hpc] and [https://github.com/rokroskar/sparkhpc] contain most of the material necessary for this, but it may be good to incorporate this in Spark directly so that Java and Scala are also well supported.
Bug: Bump Breeze from 1.0 to 1.2
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 04/May/21 10:16
Affects Version/s: 3.2.0

Breeze 1.1 release notes

[https://groups.google.com/g/scala-breeze/c/PwGpObQNE8Q]

Breeze 1.2 release notes

[https://groups.google.com/g/scala-breeze/c/u5Ge52B36u0]
Bug: Improve performance for ApplyFunctionExpression
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 10/May/21 17:39
Affects Version/s: 3.2.0

When the `ScalarFunction` itself is trivial, `ApplyFunctionExpression` could incur significant runtime cost with `zipWithIndex` call. This proposes to move the call outside the loop over each input row.
Bug: Add benchmark results for BLASBenchmark created by Github Actions machines
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 04/May/21 07:33
Affects Version/s: 3.2.0

In SPARK-34950, benchmark results were updated to the ones created by Github Actions machines.
 
The goal of this Jira issue is to add benchmark results for BLASBenchmark (added at SPARK-33882 and SPARK-35150).
Bug: Turn compilation warnings related to `procedure syntax` to errors in Scala 2.13
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 31/May/21 09:54
Affects Version/s: 3.2.0

We fixed similar compilation warnings several times（SPARK-29291， SPARK-33352 and SPARK-35526), in order to prevent the recurrence of similar problems, add a compile arg to convert `procedure syntax` related compile warnings to compile errors.
Bug: Restore to_koalas to keep the backward compatibility
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 27/May/21 03:11
Affects Version/s: 3.2.0

In https://github.com/apache/spark/pull/32516 we removed the to_koalas but we'd better to keep them for backward compatibility.

Also we need to raise proper deprecated warning for to_koalas after restore them.
Bug: Fix rounding error in DifferentiableLossAggregatorSuite with Java 11
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 26/May/21 07:16
Affects Version/s: 3.2.0



I'm personally checking whether all the tests pass with Java 11 for the current master and I found DifferentiableLossAggregatorSuite fails.
https://github.com/sarutak/spark/runs/2661859541?check_suite_focus=true#step:9:13895

The reason seems that the implementation of Blas.daxpy is different between for Java 8 and Java 11. For Java 11, Math.fma is used.

https://github.com/luhenry/netlib/blob/v2.2.0/blas/src/main/java/dev/ludovic/netlib/blas/Java8BLAS.java#L92
https://github.com/luhenry/netlib/blob/0053ea30b11686336cbdb8c7fceb41d59d268fa2/blas/src/main/java/dev/ludovic/netlib/blas/Java11BLAS.java#L40
Bug: Introduce a util function to check whether the underlying expressions of the columns are the same.
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 26/May/21 18:48
Affects Version/s: 3.2.0

In pandas on Spark, there are some places checking the underlying expressions of columns are the same or not, but it's done one-by-one. We should introduce a util function for it.
Bug: Fix HiveExternalCatalogVersionsSuite to pass with Java 11
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 26/May/21 04:27
Affects Version/s: 3.2.0

I'm personally checking whether all the tests pass with Java 11 for the current master and I found HiveExternalCatalogVersionsSuite fails.
The reason is that Spark 3.0.2 and 3.1.1 doesn't accept 2.3.8 as a hive metastore version.

HiveExternalCatalogVersionsSuite downloads Spark releases from https://dist.apache.org/repos/dist/release/spark/ and run test for each release. The Spark releases are 3.0.2 and 3.1.1 for now.

With Java 11, the suite runs with a hive metastore version which corresponds to the builtin Hive version and it's 2.3.8 for the current master.

But branch-3.0 and branch-3.1 doesn't accept 2.3.8, the suite with Java 11 fails.
Bug: skip duplicate queries in the TPCDS-related tests
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 12/May/21 13:04
Affects Version/s: 3.2.0

This ticket proposes to skip the "q6", "q34", "q64", "q74", "q75", "q78" queries in the TPCDS-related tests because the TPCDS v2.7 queries have almost the same ones; the only differences in these queries are ORDER BY columns.
Bug: Upgrade Jackson to 2.12.3
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 28/May/21 02:22
Affects Version/s: 3.2.0

Bring some bug fixes like:
 * [#3108|https://github.com/FasterXML/jackson-databind/issues/3108]: {{TypeFactory}} cannot convert {{Collection}} sub-type without type parameters to canonical form and back
 * [#207|https://github.com/FasterXML/jackson-modules-java8/issues/207]: Fail to serialize {{TemporalAdjuster}} type with 2.12

 More details:

[https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.12.3]

 

And the master branch of avro has upgrade Jackson to 2.12.3
Bug: Support propagate empty relation through aggregate
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 19/May/21 09:49
Affects Version/s: 3.2.0

The Aggregate in AQE is different with others, the `LogicalQueryStage` looks like `LogicalQueryStage(Aggregate, BaseAggregate)`. We should handle this case specially.


Logically, if the Aggregate grouping expression is not empty, we can eliminate it safely.
Bug: improve execution performance in insert...select...limit  case
Status: In Progress | Priority: Minor | Resolution: 
Created: 09/May/21 06:19
Affects Version/s: 3.0.0

In the case of `insert into...select...limit` , `CollectLimitExec` has better execution performance than `GlobalLimit` .

Before:
{code:java}
== Physical Plan ==
 Execute InsertIntoHadoopFsRelationCommand ...
 +- *(2) GlobalLimit 5
 +- Exchange SinglePartition, true, id=#39
 +- *(1) LocalLimit 5
 +- *(1) ColumnarToRow
 +- FileScan ...
{code}
After:
{code:java}
== Physical Plan ==
 Execute InsertIntoHadoopFsRelationCommand ...
 +- CollectLimit 5
 +- *(1) ColumnarToRow
 +- FileScan ....
{code}
Bug: Pin jinja2 in spark-rm/Dockerfile and add as a required dependency in the release README.md
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 17/May/21 12:33
Affects Version/s: 3.0.2

SPARK-35375 confined the version of Jinja to <3.0.0.
So it's good to note about it in docs/README.md
Bug: Upgrade Scala 2.13 to 2.13.7
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/May/21 06:05
Affects Version/s: 3.3.0

This issue aims to upgrade to Scala 2.13.7.

Scala 2.13.6 released（https://github.com/scala/scala/releases/tag/v2.13.6）. However, we skip 2.13.6 because there is a breaking behavior change at 2.13.6 which is different from both Scala 2.13.5 and Scala 3.
- https://github.com/scala/bug/issues/12403
{code}
scala3-3.0.0:$ bin/scala
scala> Array.empty[Double].intersect(Array(0.0))
val res0: Array[Double] = Array()

scala-2.13.6:$ bin/scala
Welcome to Scala 2.13.6 (OpenJDK 64-Bit Server VM, Java 1.8.0_292).
Type in expressions for evaluation. Or try :help.

scala> Array.empty[Double].intersect(Array(0.0))
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to [D
  ... 32 elided
{code}
Bug: Cross-building docker images to ARM64 is failing (with Ubuntu host)
Status: Resolved | Priority: Minor | Resolution: Not A Problem
Created: 08/May/21 16:02
Affects Version/s: 3.1.1

I was trying to cross-build Spark 3.1.1 for ARM64 so that I could deploy to a Raspberry Pi Kubernetes cluster this weekend and the Docker build fails.

Here are the commands I used:
{code:java}
docker buildx create --use
./bin/docker-image-tool.sh -n -r andygrove -t 3.1.1 -X build {code}
The Docker build for ARM64 fails on the following command:
{code:java}
 apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps{code}
The install fails with "Error while loading /usr/sbin/dpkg-split: No such file or directory".

Here is a fragment of the output showing the relevant error message.
{code:java}
#6 6.034 Get:35 https://deb.debian.org/debian buster/main arm64 libnss3 arm64 2:3.42.1-1+deb10u3 [1082 kB]
#6 6.102 Get:36 https://deb.debian.org/debian buster/main arm64 psmisc arm64 23.2-1 [122 kB]
#6 6.109 Get:37 https://deb.debian.org/debian buster/main arm64 tini arm64 0.18.0-1 [194 kB]
#6 6.767 debconf: delaying package configuration, since apt-utils is not installed
#6 6.883 Fetched 18.1 MB in 1s (13.4 MB/s)
#6 6.956 Error while loading /usr/sbin/dpkg-split: No such file or directory
#6 6.959 Error while loading /usr/sbin/dpkg-deb: No such file or directory
#6 6.961 dpkg: error processing archive /tmp/apt-dpkg-install-NdOR40/00-libncurses6_6.1+20181013-2+deb10u2_arm64.deb (--unpack):
 {code}
My host environment details:
 * Ubuntu 18.04.5 LTS
 * Docker version 20.10.6, build 370c289
Bug: The output of PCA is inconsistent
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 17/May/21 09:49
Affects Version/s: 3.1.1

1. The example from doc

 
{code:java}
import org.apache.spark.ml.feature.PCA
import org.apache.spark.ml.linalg.Vectors

val data = Array(
  Vectors.sparse(5, Seq((1, 1.0), (3, 7.0))),
  Vectors.dense(2.0, 0.0, 3.0, 4.0, 5.0),
  Vectors.dense(4.0, 0.0, 0.0, 6.0, 7.0)
)
val df = spark.createDataFrame(data.map(Tuple1.apply)).toDF("features")

val pca = new PCA()
  .setInputCol("features")
  .setOutputCol("pcaFeatures")
  .setK(3)
  .fit(df)

val result = pca.transform(df).select("pcaFeatures")
result.show(false)
{code}
 

 

the output show:
{code:java}
+-----------------------------------------------------------+
|pcaFeatures                                                |
+-----------------------------------------------------------+
|[1.6485728230883807,-4.013282700516296,-5.524543751369388] |
|[-4.645104331781534,-1.1167972663619026,-5.524543751369387]|
|[-6.428880535676489,-5.337951427775355,-5.524543751369389] |
+-----------------------------------------------------------+
{code}
2. change the Vector format

I modified the code from "Vectors.sparse(5, Seq((1, 1.0), (3, 7.0)))" to "Vectors.dense(0.0,1.0,0.0,7.0,0.0)" 。

but the output show：
{code:java}
+------------------------------------------------------------+
|pcaFeatures                                                 |
+------------------------------------------------------------+
|[1.6485728230883814,-4.0132827005162985,-1.0091435193998504]|
|[-4.645104331781533,-1.1167972663619048,-1.0091435193998501]|
|[-6.428880535676488,-5.337951427775359,-1.009143519399851]  |
+------------------------------------------------------------+
{code}
It's strange that the two outputs are inconsistent. Why?

Thanks.
Bug: Improve performance for InvokeLike.invoke
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 12/May/21 06:38
Affects Version/s: 3.2.0

`InvokeLike.invoke` uses `map` to evaluate arguments:
{code:java}
val args = arguments.map(e => e.eval(input).asInstanceOf[Object])
if (needNullCheck && args.exists(_ == null)) {
  // return null if one of arguments is null
  null
} else { 
{code}
which seems pretty expensive if the method itself is trivial. We can change it to a plain for-loop.
Bug: Avoid redundant computation in retrieval of approximate quantiles
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 28/May/21 18:36
Affects Version/s: 3.1.3

QuantileSummaries.query() is called repeatedly to obtain the approximate quantiles (e.g., in approximate_percentiles([0, 0.1, 0.5, 0.9])) but performs redundant computation in each invocation and repeated passes over the quantile sketch. These redundant computations can be eliminated by providing a version of query() that accepts the array of percentiles.