av9ash/GitBugs huggingface dataset:
Bug: spark3.1.1 use too long time to analyze table fields
Status: Resolved | Priority: Major | Resolution: Invalid
Created: 11/May/21 03:35
Affects Version/s: 3.1.1

I have a big sql with a few width tables join and complex logic, when I run that in spark 2.4 , it will take 20 minues in analyze phase, when I use spark 3.1.1, it will use about 40 minutes,

I need set spark.sql.analyzer.maxIterations=1000 in spark3.1.1.

or spark.sql.optimizer.maxIterations=1000 in spark2.4.

no other special setting for this .

I check on the spark ui , I find that there is no job generated, all executor have no active tasks, and when I set log level to debug, I find that the job is in analyze phase, analyze the fields reference.

this phase use too long time.
Bug: Spark should be more resilient to intermittent K8s flakiness
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/May/21 09:47
Affects Version/s: 3.2.0

Internal K8s errors such as an etcdserver leader election is propagated to the API client and could cause serious issues in Spark, like:

{noformat}
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at:
https://kubernetes.default.svc/api/v1/namespaces/dex-app-bl24w4z9/pods/sparkpi-10-fcd3f6781a874212-driver. Message: etcdserver: 
leader changed. Received status: Status(apiVersion=v1, code=500, details=null, kind=Status, message=etcdserver: leader changed, 
metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=null, 
status=Failure, additionalProperties={}).
{noformat}

First I try to fix in kubernetes-client by adding retries with exponential backoff:
https://github.com/fabric8io/kubernetes-client/issues/3087

If I manage it then this will could be just version update and introducing some new configs in Spark.
Bug: Handle the COUNT bug when rewriting lateral subqueries
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 28/May/21 04:04
Affects Version/s: 3.2.0

Similar to SPARK-15370, When a lateral join is rewritten as a normal join, it can lead to incorrect result since the rewrite can potentially change the semantics of the query plan. Please see SPARK-15370  for more details.
Bug: pandas API on Spark: Separate basic operations into data type based structures
Status: Resolved | Priority: Major | Resolution: Done
Created: 07/May/21 16:45
Affects Version/s: 3.2.0

Currently, the same basic operation of all data types is defined in one function, so it’s difficult to extend the behavior change based on the data types. For example, the binary operation Series + Series behaves differently based on the data type, e.g., just adding for numerical operands, concatenating for string operands, etc. The behavior difference is done by if-else in the function, so it’s messy and difficult to maintain or reuse the logic.

We should provide an infrastructure to manage the differences in these operations.

Please refer to [pandas APIs on Spark: Separate basic operations into data type based structures|https://docs.google.com/document/d/12MS6xK0hETYmrcl5b9pX5lgV4FmGVfpmcSKq--_oQlc/edit?usp=sharing] for details.
Bug: Standardize TypeError messages for unsupported basic operations
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/May/21 16:51
Affects Version/s: 3.2.0

Inconsistent TypeError messages are shown for unsupported data-type-based basic operations.

Take addition's TypeError messages for example: 
{code:java}
addition can not be applied to given types.
string addition can only be applied to string series or literals.
{code}
Standardizing TypeError messages would improve user experience and reduce maintenance costs.
Bug: Improve unit tests for data-type-based basic operations
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/May/21 16:49
Affects Version/s: 3.2.0

Unit tests for arithmetic operations are scattered in the codebase: 
 * pyspark/pandas/tests/test_ops_on_diff_frames.py
 * pyspark/pandas/tests/test_dataframe.py
 * pyspark/pandas/tests/test_series.py
 * (Upcoming) pyspark/pandas/tests/data_type_ops/

We wanted to consolidate them.

The code would be cleaner and easier to maintain.
Bug: pandas API on Spark: Enable mypy check "disallow_untyped_defs" for main codes.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 20/May/21 22:38
Affects Version/s: 3.2.0

Currently many functions in the main codes are still missing type annotations and disabled {{mypy}} check "disallow_untyped_defs".

We should add more type annotations and enable the {{mypy}} check.
Bug: Support creating a Column of numpy literal value in pandas-on-Spark
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/May/21 20:39
Affects Version/s: 3.2.0

Spark (`lit` function defined in `pyspark.sql.functions`) doesn't support creating a Column out of numpy literal value.


So `lit` function defined in `pyspark.pandas.spark.functions` should be adjusted in order to support that in pandas-on-Spark.
Bug: Migrate ALTER TABLE commands that alter columns to new framework
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 13/May/21 23:43
Affects Version/s: 3.2.0

Migrate the following ALTER TABLE commands:
 * ALTER TABLE ... ADD COLUMNS
 * ALTER TABLE ... REPLACE COLUMNS
 * ALTER TABLE ... ALTER COLUMN
 * ALTER TABLE ... RENAME COLUMN
 * ALTER TABLE ... DROP COLUMNS
Bug: Memory leak in ExecutorAllocationListener breaks dynamic allocation under high load
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/May/21 20:31
Affects Version/s: 3.1.1

ExecutorAllocationListener doesn't clean up data properly. ExecutorAllocationListener performs progressively slower and eventually fails to process events in time.

There are two problems:
 * a bug (typo?) in totalRunningTasksPerResourceProfile() method
 getOrElseUpdate() is used instead of getOrElse().
 If spark-dynamic-executor-allocation thread calls schedule() after a SparkListenerTaskEnd event for the last task in a stage
 but before SparkListenerStageCompleted event for the stage, then stageAttemptToNumRunningTask will not be cleaned up properly.
 * resourceProfileIdToStageAttempt clean-up is broken
 If a SparkListenerTaskEnd event for the last task in a stage was processed before SparkListenerStageCompleted for that stage,
 then resourceProfileIdToStageAttempt will not be cleaned up properly.

 

Bugs were introduced in this commit: https://github.com/apache/spark/commit/496f6ac86001d284cbfb7488a63dd3a168919c0f .

Steps to reproduce:
 # Launch standalone master and worker with 'spark.shuffle.service.enabled=true'
 # Run spark-shell with --conf 'spark.shuffle.service.enabled=true' --conf 'spark.dynamicAllocation.enabled=true' and paste this script
{code:java}
for (_ <- 0 until 10) {
    Seq(1, 2, 3, 4, 5).toDF.repartition(100).agg("value" -> "sum").show()
}
{code}

 # make a heap dump and examine ExecutorAllocationListener.totalRunningTasksPerResourceProfile and ExecutorAllocationListener.resourceProfileIdToStageAttempt fields

Expected: totalRunningTasksPerResourceProfile and resourceProfileIdToStageAttempt(defaultResourceProfileId) are empty
Actual: totalRunningTasksPerResourceProfile and resourceProfileIdToStageAttempt(defaultResourceProfileId) contain non-relevant data
Bug: Split SubqueryExpression's children field into outer attributes and join conditions
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/May/21 21:05
Affects Version/s: 3.2.0

Currently the children field of a subquery expression is used to store both collected outer references inside the subquery plan, and also join conditions after correlated predicates are pulled up. For example

SELECT (SELECT max(c1) FROM t1 WHERE t1.c1 = t2.c1) FROM t2

After analysis phase:

scalar-subquery [t2.c1]

After PullUpCorrelatedPredicates:

scalar-subquery [t1.c1 = t2.c1]

The references for a subquery expressions is also confusing: 

override lazy val references: AttributeSet =
 if (plan.resolved) super.references -- plan.outputSet else super.references 

We should split this children field into outer attribute references and join conditions.
Bug: Add a config for ignoring metadata directory of file stream sink
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 30/May/21 08:57
Affects Version/s: 3.2.0

FileStreamSink produces a metadata directory which logs output files per micro-batch. When we read from the output path, Spark will look at the metadata and ignore other files not in the log.

Normally it works well. But for some use-cases, we may need to ignore the metadata when reading the output path. For example, when we change the streaming query and must to run it with new checkpoint directory, we cannot use previous metadata. If we create a new metadata too, when we read the output path later in Spark, Spark only reads the files listed in the new metadata. The files written before we use new checkpoint and metadata are ignored by Spark.

Although seems we can output to different output directory every time, but it is bad idea as we will produce many directories unnecessarily.

Seems we need a config for ignoring the metadata of FileStreamSink when reading the output path.
Bug: Introduce DecimalOps
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/May/21 16:56
Affects Version/s: 3.2.0

In PySpark, {{DoubleType}} and {{FloatType}} could have NaN, whereas {{DecimalType}} not.

Now, {{DecimalType,}} {{DoubleType,}} and {{FloatType}} data share the {{FractionalOps}} class.

We would like to introduce the {{DecimalOps}} for NaN check.
Bug: Remove commons-httpclient due to EOL and CVEs
Status: Closed | Priority: Major | Resolution: Duplicate
Created: 17/May/21 22:21
Affects Version/s: 3.0.0

Spark is pulling in commons-httpclient as a dependency directly. See dependency:tree:
{code:java}
 ./build/mvn dependency:tree | grep -i "commons-httpclient"                                                                                       
Using `mvn` from path: /Users/sumeet.gajjar/cloudera/upstream-spark/build/apache-maven-3.6.3/bin/mvn
[INFO] +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] |  +- commons-httpclient:commons-httpclient:jar:3.1:provided
{code}
commons-httpclient went EOL years ago and there are most likely CVEs not being reported against it, thus we should remove it.
Bug: Children subexpr should come first than parent subexpr in subexpression elimination
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/May/21 03:44
Affects Version/s: 3.2.0

EquivalentExpressions maintains a map of equivalent expressions. It is HashMap now so the insertion order is not guaranteed to be preserved later. Subexpression elimination relies on retrieving subexpressions from the map. If there is child-parent relationships among the subexpressions, we want the child expressions come first than parent expressions, so we can replace child expressions in parent expressions with subexpression evaluation.

For example, we have two different expressions Add(Literal(1), Literal(2)) and Add(Literal(3), add).

Case 1: child subexpr comes first. Replacing HashMap with LinkedHashMap can deal with it.

addExprTree(add)
addExprTree(Add(Literal(3), add))
addExprTree(Add(Literal(3), add))

Case 2: parent subexpr comes first. For this case, we need to sort equivalent expressions.

addExprTree(Add(Literal(3), add))  => We add `Add(Literal(3), add)` into the map first, then add `add` into the map
addExprTree(add)
addExprTree(Add(Literal(3), add))
Bug: Dataset.observe fails with an assertion
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 03/May/21 15:47
Affects Version/s: 3.1.1

I hit this assertion error when using dataset.observe:
{code}
java.lang.AssertionError: assertion failed
	at scala.Predef$.assert(Predef.scala:208) ~[scala-library-2.12.10.jar:?]
	at org.apache.spark.sql.execution.AggregatingAccumulator.setState(AggregatingAccumulator.scala:204) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.sql.execution.CollectMetricsExec.$anonfun$doExecute$2(CollectMetricsExec.scala:72) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.sql.execution.CollectMetricsExec.$anonfun$doExecute$2$adapted(CollectMetricsExec.scala:71) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.TaskContext$$anon$1.onTaskCompletion(TaskContext.scala:125) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.TaskContextImpl.$anonfun$markTaskCompleted$1(TaskContextImpl.scala:124) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.TaskContextImpl.$anonfun$markTaskCompleted$1$adapted(TaskContextImpl.scala:124) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.TaskContextImpl.$anonfun$invokeListeners$1(TaskContextImpl.scala:137) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.TaskContextImpl.$anonfun$invokeListeners$1$adapted(TaskContextImpl.scala:135) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) ~[scala-library-2.12.10.jar:?]
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) ~[scala-library-2.12.10.jar:?]
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) ~[scala-library-2.12.10.jar:?]
	at org.apache.spark.TaskContextImpl.invokeListeners(TaskContextImpl.scala:135) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:124) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.scheduler.Task.run(Task.scala:147) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497) ~[spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439) [spark-core_2.12-3.1.1.jar:3.1.1]
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500) [spark-core_2.12-3.1.1.jar:3.1.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_282]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_282]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
{code}

A workaround, that I used was to add .coalesce(1) before calling this method.

It happens in a quite complex query and I have not been able to reproduce this with a simpler query

Added an screenshot of the debugger, at the moment of exception
 !2021-05-03_18-34.png!
Bug: Register merge status even after shuffle dependency is merge finalized
Status: Open | Priority: Major | Resolution: 
Created: 28/May/21 01:15
Affects Version/s: 3.1.0

Currently the merge statuses which arrive late from the external shuffle services (or shuffle mergers) won't get registered once the shuffle dependency merge is finalized.
This needs to be carefully done as there are lot of corner cases like:
a) executor/node loss causing re-computation due to fetch failure and if the merge statuses gets registered very late then that can cause inconsistencies.
b) similar such cases

cc [~mridulm80] [~mshen]
Bug: Introduce BooleanExtensionOps
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/May/21 16:55
Affects Version/s: 3.2.0

{{Now ___and____, __or____,_ ___rand____, and __ror___ are not data type based.}}

So we would like to introduce these operators for Boolean spark type IndexOps.

These bitwise operators should be able to apply to other data types classes. However, this is not the goal of this PR.

extension_dtypes process these operators differently from the rest of the types, so BooleanExtensionOps is introduced.
Bug: Make the conversion from/to pandas data-type-based for non-ExtensionDtypes
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/May/21 20:38
Affects Version/s: 3.2.0

The conversion from/to pandas includes logic for checking data types and behaving accordingly.

That makes code hard to change or maintain.

Since we have introduced the Ops class per non-ExtensionDtypes data type, we ought to make the conversion from/to pandas data-type-based.

Ops class per ExtensionDtype and its data-type-based from/to pandas will be implemented in a separate PR.
Bug: Apply black to pandas API on Spark codes.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/May/21 07:36
Affects Version/s: 3.2.0

Make it easier and more efficient to static analysis, we'd better to apply `black` to the pandas API on Spark.

Koalas project is using black for [reformatting script|https://github.com/databricks/koalas/blob/master/dev/reformat].
Bug: UnsupportedOperationException: WholeStageCodegen (3) does not implement doExecuteBroadcast
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/May/21 00:09
Affects Version/s: 3.2.0

How to reproduce:
{code:scala}
sql(
      """
        |SELECT s.store_id, f.product_id
        |FROM (SELECT DISTINCT * FROM fact_sk) f
        |  JOIN (SELECT
        |          *,
        |          ROW_NUMBER() OVER (PARTITION BY store_id ORDER BY state_province DESC) AS rn
        |        FROM dim_store) s
        |   ON f.store_id = s.store_id
        |WHERE s.country = 'DE' AND s.rn = 1
        |""".stripMargin).show
{code}

{noformat}
Caused by: java.lang.UnsupportedOperationException: WholeStageCodegen (3) does not implement doExecuteBroadcast
	at org.apache.spark.sql.execution.SparkPlan.doExecuteBroadcast(SparkPlan.scala:297)
	at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.doExecuteBroadcast(AdaptiveSparkPlanExec.scala:323)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeBroadcast$1(SparkPlan.scala:192)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:217)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:214)
	at org.apache.spark.sql.execution.SparkPlan.executeBroadcast(SparkPlan.scala:188)
{noformat}
Bug: Do not drop cached RDD blocks to accommodate blocks from decommissioned block manager if enough memory is not available
Status: In Progress | Priority: Major | Resolution: 
Created: 26/May/21 10:27
Affects Version/s: 3.1.1

In current block manager decommissioning flow, existing cached blocks in memory are dropped if enough memory is not available to accommodate blocks from decommissioned block manager.

 

Why should blocks from a decommissioned block manager have more priority than an already cached block? 

We should place blocks from decommission block manager on a peer block manager only when enough memory is available
Bug: Support subexpression elimination for higher order functions
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/May/21 19:07
Affects Version/s: 3.1.1

Currently higher order functions are not candidates for subexpression elimination. This is because all higher order functions have different semantic hashes, due to "exprId" and "value" in "NamedLambdaVariable". These always are unique, so the semanticHash of a NamedLambdaVariable is always unique. Also, [https://github.com/apache/spark/pull/32424] might throw a wrench in things some too, depending on how you define your expressions the name could be different.
{code:java}
scala> var d = transform($"a", x => x + 1)
d: org.apache.spark.sql.Column = transform(a, lambdafunction((x_2 + 1), x_2))

scala> var e = transform($"a", x => x + 1)
e: org.apache.spark.sql.Column = transform(a, lambdafunction((x_3 + 1), x_3))

scala> struct(d.alias("1"), d.alias("2")).expr
res9: org.apache.spark.sql.catalyst.expressions.Expression = struct(NamePlaceholder, transform('a, lambdafunction((lambda 'x_2 + 1), lambda 'x_2, false)) AS 1#4, NamePlaceholder, transform('a, lambdafunction((lambda 'x_2 + 1), lambda 'x_2, false)) AS 2#5)

scala> struct(d.alias("1"), e.alias("2")).expr
res10: org.apache.spark.sql.catalyst.expressions.Expression = struct(NamePlaceholder, transform('a, lambdafunction((lambda 'x_2 + 1), lambda 'x_2, false)) AS 1#6, NamePlaceholder, transform('a, lambdafunction((lambda 'x_3 + 1), lambda 'x_3, false)) AS 2#7)
{code}
Bug: Remove redundant subexpression evaluation in nested subexpressions
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 28/May/21 23:11
Affects Version/s: 3.2.0

In some cases, whole-stage codegen's subexpression elimination will do redundant subexpression evaluation. We should reduce it. For example, if we have two sub-exprs:

1. simpleUDF($"id")
2. functions.length(simpleUDF($"id"))


We should only evaluate `simpleUDF($"id")` once, i.e.

subExpr1 = simpleUDF($"id");
subExpr2 = functions.length(subExpr1);
Bug: Support arithmetic operations against bool IndexOpsMixin
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/May/21 22:13
Affects Version/s: 3.2.0

Existing binary operations of bool Series in Koalas do not match pandas’ behaviors.

pandas take True as 1, False as 0 when dealing with numeric values, numeric collections, and numeric Series; whereas Koalas raises an AnalysisException no matter what the binary operation is.

We aim to match pandas' behaviors.
Bug: Redact the sensitive info in the result of Set command
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/May/21 10:51
Affects Version/s: 1.6.3

Currently, the results of following SQL queries are not redacted:
```
SET [KEY];
SET;
```
For example:


{code:java}
scala> spark.sql("set javax.jdo.option.ConnectionPassword=123456").show()
+--------------------+------+
|                 key| value|
+--------------------+------+
|javax.jdo.option....|123456|
+--------------------+------+


scala> spark.sql("set javax.jdo.option.ConnectionPassword").show()
+--------------------+------+
|                 key| value|
+--------------------+------+
|javax.jdo.option....|123456|
+--------------------+------+


scala> spark.sql("set").show()
+--------------------+--------------------+
|                 key|               value|
+--------------------+--------------------+
|javax.jdo.option....|              123456|

{code}

We should hide the sensitive information and redact the query output.
Bug: Ambiguous self-join doesn't fail after transfroming the dataset to dataframe
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 20/May/21 03:27
Affects Version/s: 3.1.0

{code:java}
test("SPARK-28344: fail ambiguous self join - Dataset.colRegex as column ref") {
  val df1 = spark.range(3)
  val df2 = df1.filter($"id" > 0)

  withSQLConf(
    SQLConf.FAIL_AMBIGUOUS_SELF_JOIN_ENABLED.key -> "true",
    SQLConf.CROSS_JOINS_ENABLED.key -> "true") {
    assertAmbiguousSelfJoin(df1.join(df2, df1.colRegex("id") > df2.colRegex("id")))
  }
}
{code}
For this unit test, if we append `.toDF()` to both df1 and df2, the query won't fail.
Bug: Support outer references in Aggregate
Status: Open | Priority: Major | Resolution: 
Created: 28/May/21 04:48
Affects Version/s: 3.2.0

Support Aggregate to host outer query references. For example:
{code:sql}
CREATE VIEW t(c1, c2) AS VALUES (0, 1), (1, 2)

-- scalar subquery example
SELECT (SELECT SUM(c1)) FROM t

-- lateral join example
SELECT * FROM t, LATERAL (SELECT SUM(c1))
{code}

Currently Spark throws this error:
{code}
org.apache.spark.sql.AnalysisException: Expressions referencing the outer query are not supported outside of WHERE/HAVING clauses:
Aggregate [sum(outer(c1#23)) AS sum(outer(spark_catalog.default.t.c1))#28L]
+- OneRowRelation
{code}

Postgres SQL test example (groupingsets.sql)
{code}
select * from (values (1),(2)) v(a) left join lateral (select v.a, four, ten, count(*) from onek group by cube(four,ten)) s on true order by v.a,four,ten

org.apache.spark.sql.AnalysisException
Expressions referencing the outer query are not supported outside of WHERE/HAVING clauses:
Aggregate [four#x, ten#x, spark_grouping_id#xL], [outer(a#x), four#x, ten#x, count(1) AS count(1)#xL]
{code}