av9ash/GitBugs huggingface dataset:
Bug: pyspark hangs in local mode running rdd map operation
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 28/Mar/22 17:47
Affects Version/s: 3.2.1

In spark 3.2.1 (spark 3.2.0 doesn't show this issue), pyspark will hang when running and RDD map operations and converting to a dataframe.  Code is below to reproduce.  

Env:
spark 3.2.1 local mode, just run {{./bin/pyspark --driver-memory XXXXG --driver-cores XXXX}}

{{download dataset from here [https://rapidsai-data.s3.us-east-2.amazonaws.com/spark/mortgage.zip]}}
{{just 200000 rows could reproduce the issue }}{{head -n 200000 mortgage_eval_merged.csv > mortgage_eval_merged-small.csv}}{{{} but if the input dataset is small, such 50000 rows, it works well.{}}}{{{{}}{}}}run codes below:
{code:java}
path = "/XXXX/mortgage_eval_merged-small.csv"
src_data = sc.textFile(path).map(lambda x:x.split(","))
column_list =['c1','c2','c3','c4','c5','c6','c7','c8','c9','c10','c11','c12','c13','c14','c15','c16','c17','c18','c19','c20','c21','c22','c23','c24','c25','c26','c27','c28']
df = spark.createDataFrame(src_data,column_list)
print(df.show(1)){code}
Bug: Stream-stream outer join has a possible correctness issue due to weakly read consistent on outer iterators
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 29/Mar/22 09:22
Affects Version/s: 3.2.1

We figured out stream-stream join has the same issue with SPARK-38320 on the appended iterators. Since the root cause is same as SPARK-38320, this is only reproducible with RocksDB state store provider, but even with HDFS backed state store provider, it is not guaranteed by interface contract hence may depend on the JVM vendor, version, etc.

I can easily construct the scenario of “data loss” in state store.

Condition follows:
 * Use stream-stream time interval outer join

 ** left outer join has an issue on left side, right outer join has an issue on right side, full outer join has an issue on both sides

 * At batch N, produce row(s) on the problematic side which are non-late

 * At the same batch (batch N), some row(s) on the problematic side should be evicted by watermark condition

When the condition is fulfilled, out of sync happens with keyToNumValues between state and the iterator in evict phase. If eviction of the row happens for the grouping key (updating keyToNumValues), the eviction phase “overwrites” keyToNumValues in the state as the value it calculates.

Given that the eviction phase “do not know” about the new rows (keyToNumValues is out of sync), effectively discarding all rows from the state being added in the batch N.
Bug: Revert the parser changes for DEFAULT column support
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 16/Mar/22 10:59
Affects Version/s: 3.3.0

Revert the commit https://github.com/apache/spark/commit/e21cb62d02c85a66771822cdd49c49dbb3e44502 from branch-3.3.
Bug: Arbitrary shell command injection via Utils.unpack()
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 23/Mar/22 03:53
Affects Version/s: 3.1.2

There is a risk for arbitrary shell command injection via {{Utils.unpack}} when the filename is controlled by a malicious user. This is due to an issue in Hadoop's {{unTar}}, that is not properly escaping the filename before passing to a shell command:https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java#L904
Bug: ceil and floor return different types when called from scala than sql
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 19/Mar/22 13:03
Affects Version/s: 3.3.0

In Spark 3.3.0  SPARK-37475 [PR|http://example.com/][https://github.com/apache/spark/pull/34729] went in and added support for a scale parameter to floor and ceil.  There was [discussion|https://github.com/apache/spark/pull/34729#discussion_r761157050] about potential incompatibilities, specifically with respect to the return types. It looks like it was [decided|https://github.com/apache/spark/pull/34729#discussion_r767446855 to keep the old behavior if no scale parameter was passed in, but use the new functionality if a scale is passed in.

 

But the scala API didn't get updated to do the same thing as the SQL API.
{code:scala}
scala> spark.range(1).selectExpr("id", "ceil(id) as one_arg_sql", "ceil(id, 0) as two_arg_sql").select(col("*"), ceil(col("id")).alias("one_arg_func"), ceil(col("id"), lit(0)).alias("two_arg_func")).printSchema
root
 |-- id: long (nullable = false)
 |-- one_arg_sql: long (nullable = true)
 |-- two_arg_sql: decimal(20,0) (nullable = true)
 |-- one_arg_func: decimal(20,0) (nullable = true)
 |-- two_arg_func: decimal(20,0) (nullable = true)
 

scala> spark.range(1).selectExpr("cast(id as double) as id").selectExpr("id", "ceil(id) as one_arg_sql", "ceil(id, 0) as two_arg_sql").select(col("*"), ceil(col("id")).alias("one_arg_func"), ceil(col("id"), lit(0)).alias("two_arg_func")).printSchema
root
 |-- id: double (nullable = false)
 |-- one_arg_sql: long (nullable = true)
 |-- two_arg_sql: decimal(30,0) (nullable = true)
 |-- one_arg_func: decimal(30,0) (nullable = true)
 |-- two_arg_func: decimal(30,0) (nullable = true) {code}
And because the python code call into this too it also has the same problem. I suspect that the java and R code also expose it too, but I didn't check.
Bug: Upgrade to Py4J 0.10.9.5
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 16/Mar/22 05:24
Affects Version/s: 3.2.1

There is a resource leak bug, see https://github.com/py4j/py4j/pull/471. We should upgrade Py4J to 0.10.9.4 to fix this
Bug: Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1
Status: Resolved | Priority: Critical | Resolution: Duplicate
Created: 07/Mar/22 03:30
Affects Version/s: 3.2.1

[https://spark.apache.org/downloads.html] 

!11271646623853_.pic.jpg!
Bug: UnsafeHashedRelation should serialize numKeys out
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 13/Mar/22 12:18
Affects Version/s: 3.2.0

At present, UnsafeHashedRelation does not write out numKeys during serialization, so the numKeys of UnsafeHashedRelation obtained by deserialization is equal to 0. The numFields of UnsafeRows returned by UnsafeHashedRelation.keys() are all 0, which can lead to missing or incorrect data.

 

For example, in SubqueryBroadcastExec, the HashedRelation.keys() function is called.
{code:java}
val broadcastRelation = child.executeBroadcast[HashedRelation]().value
val (iter, expr) = if (broadcastRelation.isInstanceOf[LongHashedRelation]) {
  (broadcastRelation.keys(), HashJoin.extractKeyExprAt(buildKeys, index))
} else {
  (broadcastRelation.keys(),
    BoundReference(index, buildKeys(index).dataType, buildKeys(index).nullable))
}{code}
Bug: Volcano queue is not deleted
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 11/Mar/22 01:18
Affects Version/s: 3.3.0

{code}
$ k delete queue queue0
Error from server: admission webhook "validatequeue.volcano.sh" denied the request: only queue with state `Closed` can be deleted, queue `queue0` state is `Open`
{code}

{code}
[info] org.apache.spark.deploy.k8s.integrationtest.VolcanoSuite *** ABORTED *** (7 minutes, 40 seconds)
[info]   io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: DELETE at: https://44bea09e70a5147f6b5b347ec26de85f.gr7.us-west-2.eks.amazonaws.com/apis/scheduling.volcano.sh/v1beta1/queues/queue-2u-3g. Message: admission webhook "validatequeue.volcano.sh" denied the request: only queue with state `Closed` can be deleted, queue `queue-2u-3g` state is `Open`. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook "validatequeue.volcano.sh" denied the request: only queue with state `Closed` can be deleted, queue `queue-2u-3g` state is `Open`, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=null, status=Failure, additionalProperties={}).
{code}
Bug: ResolveFunctions implemented incorrectly requiring multiple passes to Resolve Nested Expressions 
Status: In Progress | Priority: Critical | Resolution: 
Created: 10/Mar/22 21:55
Affects Version/s: 3.2.0

ResolveFunctions Rule is implemented incorrectly requiring multiple passes to Resolve Nested Expressions:

While Plan object is traversed correctly in post-order (bottoms-up, `plan.resolveOperatorsUpWithPruning), internally, Plan children though are traversed incorrectly in pre-order (top-down, using `transformExpressionsWithPruning`):

 
{code:java}
case q: LogicalPlan =>
  q.transformExpressionsWithPruning(...) { ... } {code}
 

Traversing in pre-order means that attempt is taken to resolve the current node, before its children are resolved, which is incorrect, since the node itself could not be resolved before its children are.

While this is not leading to failures yet, this is taxing on performance – most of the expressions in Spark should be able to be resolved in a *single pass* (if resolved bottoms-up, take reproducible sample at the bottom). Instead, it currently takes Spark at least *N*  iterations to resolve such expressions, where N is proportional to the depth of the Expression tree.

 

Example to reproduce: 

 
{code:java}
def resolveExpr(spark: SparkSession, exprStr: String, tableSchema: StructType): Expression = {
  val expr = spark.sessionState.sqlParser.parseExpression(exprStr)
  val analyzer = spark.sessionState.analyzer
  val schemaFields = tableSchema.fields

  val resolvedExpr = {
    val plan: LogicalPlan = Filter(expr, LocalRelation(schemaFields.head, schemaFields.drop(1): _*))
    val rules: Seq[Rule[LogicalPlan]] = {
      analyzer.ResolveFunctions ::
      analyzer.ResolveReferences ::
      Nil
    }

    rules.foldRight(plan)((rule, plan) => rule.apply(plan))
      .asInstanceOf[Filter]
      .condition
  }
  resolvedExpr
}

// Invoke with
resolveExpr(spark, "date_format(to_timestamp(B, 'yyyy-MM-dd'), 'MM/dd/yyyy')", StructType(StructField("B", StringType))){code}
Bug: Push partial aggregation through join
Status: In Progress | Priority: Major | Resolution: 
Created: 10/Mar/22 15:09
Affects Version/s: 3.3.0

Please see https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Request-and-Transaction-Processing/Join-Planning-and-Optimization/Partial-GROUP-BY-Block-Optimization for more details.
Bug: Spark doc build not work on Mac OS M1
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 10/Mar/22 02:26
Affects Version/s: 3.3.0

 
{code:java}
diff --git a/docs/.bundle/config b/docs/.bundle/config
index b13821f801..68c1ee493a 100644
--- a/docs/.bundle/config
+++ b/docs/.bundle/config
@@ -1,2 +1,3 @@
 ---
 BUNDLE_PATH: ".local_ruby_bundle"
+BUNDLE_BUILD__FFI: "--enable-libffi-alloc"
diff --git a/docs/Gemfile b/docs/Gemfile
index f991622708..6c35201296 100644
--- a/docs/Gemfile
+++ b/docs/Gemfile
@@ -17,6 +17,7 @@
 source "https://rubygems.org"
+gem "ffi", "1.15.5"
 gem "jekyll", "4.2.1"
 gem "rouge", "3.26.0"
 gem "jekyll-redirect-from", "0.16.0"
{code}
After above patch redo `bundle install`, then it works, you could see this as ref if you meet the same issue.

will take a deep look to solve this.

 

related: https://github.com/ffi/ffi/issues/864
Bug: Repartition + Stage retries could lead to incorrect data 
Status: Open | Priority: Major | Resolution: 
Created: 02/Mar/22 08:32
Affects Version/s: 2.4.0

Spark repartition uses RoundRobinPartitioning, the generated results is non-deterministic when data has some randomness and stage/task retries happen.

The bug can be triggered when upstream data has some randomness, a repartition is called on them, then followed by result stage (could be more stages).
As the pattern shows below:
upstream stage (data with randomness) -> (repartition shuffle) -> result stage

When one executor goes down at result stage, some tasks of that stage might have finished, others would fail, shuffle files on that executor also get lost, some tasks from previous stage (upstream data generation, repartition) will need to rerun to generate dependent shuffle data files.
Because data has some randomness, regenerated data in upstream retried tasks is slightly different, repartition then generates inconsistent ordering, then tasks at result stage will be retried generating different data.

This is similar but different to https://issues.apache.org/jira/browse/SPARK-23207, fix for it uses extra local sort to make the row ordering deterministic, the sorting algorithm it uses simply compares row/record hash. But in this case, upstream data has some randomness, the sorting algorithm doesn't help keep the order, thus RoundRobinPartitioning introduced non-deterministic result.

The following code returns 986415, instead of 1000000:
{code:java}
import scala.sys.process._
import org.apache.spark.TaskContext

case class TestObject(id: Long, value: Double)

val ds = spark.range(0, 1000 * 1000, 1).repartition(100, $"id").withColumn("val", rand()).repartition(100).map { 
  row => if (TaskContext.get.stageAttemptNumber == 0 && TaskContext.get.attemptNumber == 0 && TaskContext.get.partitionId > 97) {
    throw new Exception("pkill -f java".!!)
  }
  TestObject(row.getLong(0), row.getDouble(1))
}

ds.toDF("id", "value").write.mode("overwrite").saveAsTable("tmp.test_table")

spark.sql("select count(distinct id) from tmp.test_table").show{code}
Command: 
{code:java}
spark-shell --num-executors 10 (--conf spark.dynamicAllocation.enabled=false --conf spark.shuffle.service.enabled=false){code}
To simulate the issue, disable external shuffle service is needed (if it's also enabled by default in your environment),  this is to trigger shuffle file loss and previous stage retries.
In our production, we have external shuffle service enabled, this data correctness issue happened when there were node losses.

Although there's some non-deterministic factor in upstream data, user wouldn't expect  to see incorrect result.
Bug: Spark on Kubernetes: add separate pod watcher service to reduce pressure on K8s API server
Status: Open | Priority: Major | Resolution: 
Created: 27/Mar/22 19:50
Affects Version/s: 3.2.1

Spark driver will listen to all pods events to manage its executor pods. This will cause pressure on Kubernetes API server in a large cluster, because there will be many drivers connect to the API server and watch for the pods.

 

An alternative is to have a separate service to listen and watch all pod events. Then each Spark driver only connects to that service to get pod events. This will reduce the load on Kubernetes API server.
Bug: SQL Error Attribution Framework
Status: Resolved | Priority: Major | Resolution: Implemented
Created: 21/Mar/22 15:40
Affects Version/s: 3.3.0

Currently,  there is not enough error context for runtime ANSI failures.

In the following example, the error message only tells that there is a "divide by zero" error, without pointing out where the exact SQL statement is.
{code:java}
> SELECT
  ss1.ca_county,
  ss1.d_year,
  ws2.web_sales / ws1.web_sales web_q1_q2_increase,
  ss2.store_sales / ss1.store_sales store_q1_q2_increase,
  ws3.web_sales / ws2.web_sales web_q2_q3_increase,
  ss3.store_sales / ss2.store_sales store_q2_q3_increase
FROM
  ss ss1, ss ss2, ss ss3, ws ws1, ws ws2, ws ws3
WHERE
  ss1.d_qoy = 1
    AND ss1.d_year = 2000
    AND ss1.ca_county = ss2.ca_county
    AND ss2.d_qoy = 2
    AND ss2.d_year = 2000
    AND ss2.ca_county = ss3.ca_county
    AND ss3.d_qoy = 3
    AND ss3.d_year = 2000
    AND ss1.ca_county = ws1.ca_county
    AND ws1.d_qoy = 1
    AND ws1.d_year = 2000
    AND ws1.ca_county = ws2.ca_county
    AND ws2.d_qoy = 2
    AND ws2.d_year = 2000
    AND ws1.ca_county = ws3.ca_county
    AND ws3.d_qoy = 3
    AND ws3.d_year = 2000
    AND CASE WHEN ws1.web_sales > 0
    THEN ws2.web_sales / ws1.web_sales
        ELSE NULL END
    > CASE WHEN ss1.store_sales > 0
    THEN ss2.store_sales / ss1.store_sales
      ELSE NULL END
    AND CASE WHEN ws2.web_sales > 0
    THEN ws3.web_sales / ws2.web_sales
        ELSE NULL END
    > CASE WHEN ss2.store_sales > 0
    THEN ss3.store_sales / ss2.store_sales
      ELSE NULL END
ORDER BY ss1.ca_county
 {code}
{code:java}
org.apache.spark.SparkArithmeticException: divide by zero at org.apache.spark.sql.errors.QueryExecutionErrors$.divideByZeroError(QueryExecutionErrors.scala:140) at org.apache.spark.sql.catalyst.expressions.DivModLike.eval(arithmetic.scala:437) at org.apache.spark.sql.catalyst.expressions.DivModLike.eval$(arithmetic.scala:425) at org.apache.spark.sql.catalyst.expressions.Divide.eval(arithmetic.scala:534)
...{code}
 

I suggest that we provide details in the error message,  including:
 * the problematic expression from the original SQL query, e.g. "ss3.store_sales / ss2.store_sales store_q2_q3_increase"
 * the line number and starting char position of the problematic expression, in case of queries like "select a + b from t1 union select a + b from t2"

So that the error message will be precise 
{code:java}
org.apache.spark.SparkArithmeticException: divide by zero
SparkArithmeticException: divide by zero. To return NULL instead, use 'try_divide'. If necessary set spark.sql.ansi.enabled to false (except for ANSI interval type) to bypass this error.
== SQL(line 2, position 43) ==
ws2.web_sales / ws1.web_sales web_q1_q2, ss2.store_sales / ss1.store_sales store_q1_q2
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ {code}
[^SQL Error Attribution Framework.pdf]
Bug: Refactor framework so as JDBC dialect could compile filter by self way
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/Mar/22 07:27
Affects Version/s: 3.3.0

Currently, Spark DS V2 could push down filters into JDBC source. However, only the most basic form of filter is supported.
On the other hand, some JDBC source could not compile the filters by themselves way.
Bug: Add the `DATEDIFF` and `DATE_DIFF` aliases for `TIMESTAMPDIFF()`
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Mar/22 08:54
Affects Version/s: 3.3.0

Introduce the datediff()/date_diff() function, which takes three arguments: unit, and two datetime expressions, i.e.,
{code:sql}
datediff(unit, startDatetime, endDatetime)
{code}
The function can be an alias to timestampdiff().
Bug: Complex calculations with  lead to driver oom
Status: Open | Priority: Major | Resolution: 
Created: 29/Mar/22 08:46
Affects Version/s: 3.2.0

My spark job is working fine in version 3.0.1. After I upgraded to 3.2, the driver would hang during runtime due to oom. The dump file shows that the stageMetrics in SQLAppStatusListener are taking up a lot of memory.I'm wondering if it's related to the SPARK-33016 change, or if the execution plan change has created more tasks, causing the driver to run out of memory, or some other reason.
 !screenshot-1.png!
Bug: Upgrade bcprov-jdk15on from 1.60 to 1.67
Status: Resolved | Priority: Major | Resolution: Invalid
Created: 05/Mar/22 13:48
Affects Version/s: 3.3.0

Upgrade bcprov-jdk15on from 1.60 to 1.67 [CVE-2020-15522|https://nvd.nist.gov/vuln/detail/CVE-2020-15522]

[releasenotes.|https://github.com/bcgit/bc-java/blob/master/docs/releasenotes.html]
Bug: Unify the data validation
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Mar/22 06:51
Affects Version/s: 3.4.0

1, input vector validation is missing in most algorithms, when the input dataset contains some invalid values (NaN/Infinity), then:
 * the training may run successfuly and return model containing invalid coefficients, like LinearSVC
 * the training may fail with irrelevant message, like KMeans

 
{code:java}
import org.apache.spark.ml.feature._
import org.apache.spark.ml.linalg._
import org.apache.spark.ml.classification._
import org.apache.spark.ml.clustering._
val df = sc.parallelize(Seq(LabeledPoint(1.0, Vectors.dense(1.0, Double.NaN)), LabeledPoint(0.0, Vectors.dense(Double.PositiveInfinity, 2.0)))).toDF()

val svc = new LinearSVC()
val model = svc.fit(df)

scala> model.intercept
res0: Double = NaN

scala> model.coefficients
res1: org.apache.spark.ml.linalg.Vector = [NaN,NaN]

val km = new KMeans().setK(2)
scala> km.fit(df)
22/03/17 14:29:10 ERROR Executor: Exception in task 11.0 in stage 10.0 (TID 113)
java.lang.IllegalArgumentException: requirement failed: Both norms should be greater or equal to 0.0, found norm1=NaN, norm2=Infinity
    at scala.Predef$.require(Predef.scala:281)
    at org.apache.spark.mllib.util.MLUtils$.fastSquaredDistance(MLUtils.scala:543)
{code}
 

We should make ml algorithms fail fast, if the input dataset is invalid.

 

2, there exists some methods to validate input labels and weights in different files:
 * {{org.apache.spark.ml.functions}}
 * org.apache.spark.ml.util.DatasetUtils
 * org.apache.spark.ml.util.MetadataUtils,
 * org.apache.spark.ml.Predictor
 * etc.

 

I think it is time to unify realtive methods to one source file.
Bug: DataFilter pushed down with PartitionFilter for Orc
Status: Open | Priority: Major | Resolution: 
Created: 07/Mar/22 03:24
Affects Version/s: 3.4.0

At present, for orc data source, the Filter is divided into DataFilter and PartitionFilter when it is pushed down, but when the Filter removes the PartitionFilter, it means that all Partitions will scan all DataFilter conditions, which may cause full data scan.

Based on SPARK-38041, we can pushdown dataFilter with partitionFilter to ORC, and remove partitionFilter at runtime.
Bug: ANTLR grammar definition in separate Parser and Lexer files
Status: Closed | Priority: Major | Resolution: Fixed
Created: 01/Mar/22 20:02
Affects Version/s: 3.3.0

Suggesting to separate the ANTLR grammar defined in `SqlBase.g4` into separate parser `SqlBaseParser.g4` and lexer `SqlBaseLexer.g4`. 

Benefits:

*Gain more flexibility when implementing new SQL features*

The current ANTLR grammar definition is given as a mixed grammar in the `SqlBase.g4` file.

By separating the lexer and parser, we will be able to use the full power of ANTLR parser and lexer grammars. e.g. lexer mode. This will give us more flexibility when implementing new SQL features.

*The code is more clean.* 

Having parser and lexer in different files also keeps the code more explicit about which is the parser and which is the lexer.
Bug: Add sortWithinGroups to KeyValueGroupedDataset
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Mar/22 15:49
Affects Version/s: 3.4.0

The existing methods {{KeyValueGroupedDataset.flatMapGroups}} and {{KeyValueGroupedDataset.cogroup}} provide an iterator of rows for each group key. If user code requires those rows in a particular order, that iterator would have to be sorted first, which is against the idea of an iterator in the first place. Methods {{flatMapGroups}} and {{cogroup}} have the great advantage that they work with groups that are too large to fit into memory of one executor. Sorting them in the user function breaks this property.

[org.apache.spark.sql.KeyValueGroupedDataset|https://github.com/apache/spark/blob/47485a3c2df3201c838b939e82d5b26332e2d858/sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala#L134-L137]:
{noformat}
Internally, the implementation will spill to disk if any given group is too large to fit into
memory. However, users must take care to avoid materializing the whole iterator for a group
(for example, by calling `toList`) unless they are sure that this is possible given the memory
constraints of their cluster.
{noformat}
The implementations of {{KeyValueGroupedDataset.flatMapGroups}} and {{KeyValueGroupedDataset.cogroup}} already sort each partition according to the group key. By additionally sorting by some data columns, the iterator can be guaranteed to provide some order.

New method {{KeyValueGroupedDataset.sortWithinGroups}} (similar to {{{}Dataset.sortWithinPartitions){}}}would allow to define order within the groups.