av9ash/GitBugs huggingface dataset:
Bug: IntervalUtils.add() may result in 'long overflow'
Status: Open | Priority: Major | Resolution: 
Created: 25/Feb/21 00:16
Affects Version/s: 3.0.2

I noticed the following when running test suite:

build/sbt "sql/testOnly *SQLQueryTestSuite"
{code}
19:10:17.977 ERROR org.apache.spark.scheduler.TaskSetManager: Task 1 in stage 6416.0 failed 1 times; aborting job
[info] - postgreSQL/int4.sql (2 seconds, 543 milliseconds)
19:10:20.994 ERROR org.apache.spark.executor.Executor: Exception in task 1.0 in stage 6476.0 (TID 7789)
java.lang.ArithmeticException: long overflow
    at java.lang.Math.multiplyExact(Math.java:892)
    at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.project_doConsume_0$(Unknown Source)
    at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
    at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
    at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:755)
    at org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:345)
    at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:898)
    at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:898)
    at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
{code}
{code}
19:15:38.255 ERROR org.apache.spark.executor.Executor: Exception in task 0.0 in stage 14744.0 (TID 16705)
java.lang.ArithmeticException: long overflow
    at java.lang.Math.addExact(Math.java:809)
    at org.apache.spark.sql.types.LongExactNumeric$.plus(numerics.scala:105)
    at org.apache.spark.sql.types.LongExactNumeric$.plus(numerics.scala:104)
    at org.apache.spark.sql.catalyst.expressions.Add.nullSafeEval(arithmetic.scala:268)
    at org.apache.spark.sql.catalyst.expressions.BinaryExpression.eval(Expression.scala:573)
    at org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.apply(InterpretedMutableProjection.scala:97)
{code}
This likely was caused by the following line:
{code}
    val microseconds = left.microseconds + right.microseconds
{code}
We should check whether the addition would produce overflow before adding.
Bug: Repartition miss/duplicated data
Status: Resolved | Priority: Major | Resolution: Not A Problem
Created: 25/Feb/21 11:43
Affects Version/s: 3.0.1

We have a SQL
{code:java}
INSERT OVERWRITE TABLE t1 
SELECT /*+ repartition(300) */ * from t2.{code}
Below is SQL metrics of the repartition ShuffleExchange. we can see that the shuffle record written and records read is not same. 

In the result table, there are some data missing and some data duplicated.

!image-2021-02-25-19-43-49-687.png!

!image-2021-02-25-19-46-52-809.png|width=408,height=654!!image-2021-02-25-19-47-10-005.png|width=282,height=414!

We can see that *InsertIntoHadoopFsRelationCommand's output is save as repartition Exchange's record read(reducer side)*

*and repartition Exchange's shuffle record written (mapper side written) is same as Filter's output.*

*So we can see that repartition's Exchange return wrong data.*

 

*In our env, AQE and speculation is open.*
Bug: New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/Feb/21 03:04
Affects Version/s: 3.0.0

We will build a new rpc message `FetchShuffleBlocks` when `OneForOneBlockFetcher` init in replace of `OpenBlocks` to use adaptive feature, this introduce additional problems as follows.

`OneForOneBlockFetcher` will init a `blockIds` String array to catch chunk fetch success, it will use index in `blockIds` to fetch blocks and match blockId in `blockIds` when chunk data return. So the `blockIds` 's order must be consistent with fetchChunk index, but the new `FetchShuffleBlocks` return chunk order is not same as `blockIds`.

This will lead to the return data not match the blockId,  and this can lead to data corretness when retry to fetch after fetch block chunk failed.

Fetch chunk orker code and match blockId when rerun data code as follows: 

!image-2021-02-25-11-27-34-429.png|width=446,height=251!!image-2021-02-25-11-28-31-255.png|width=445,height=159!

Howerver, the fetch order in shuffle service,

!image-2021-02-25-11-30-03-834.png|width=510,height=361!

So, it will fetch some wrong block data when chunk fetch failed beause the blocks's wrong order.

!image-2021-02-25-11-31-59-110.png|width=601,height=204!
Bug: Remove dependency to commons-httpclient, is not used and has vulnerabilities.
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 08/Feb/21 15:41
Affects Version/s: 3.1.0

<dependency>
 <groupId>commons-httpclient</groupId>
 <artifactId>commons-httpclient</artifactId>
</dependency>

 

Has vulnerabilities as below:

 

CVE-2012-6153

CVE-2012-5783

 

Also, after removing it and running `spark/sql/hive$mvn compile test` the result is SUCCESS
Bug: warehouse path should be resolved ahead of populating and use
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Feb/21 05:57
Affects Version/s: 3.0.2

Currently, the warehouse path gets fully qualified in the caller side for creating a database, table, partition, etc. An unqualified path is populated into Spark and Hadoop confs, which leads to inconsistent API behaviors.  We should make it qualified ahead.


When the value is a relative path `spark.sql.warehouse.dir=lakehouse`, for example.

If the default database is absent at runtime, the app fails with

{code:java}
Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file:./datalake
	at org.apache.hadoop.fs.Path.initialize(Path.java:263)
	at org.apache.hadoop.fs.Path.<init>(Path.java:254)
	at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:133)
	at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:137)
	at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:150)
	at org.apache.hadoop.hive.metastore.Warehouse.getDefaultDatabasePath(Warehouse.java:163)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:636)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:655)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:431)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:79)
	... 73 more
{code}

If the default database is present at runtime, the app can work with it, and if we create a database, it gets fully qualified, for example


{code:sql}
spark-sql> create database test2 location 'datalake';
21/02/26 21:52:57 WARN ObjectStore: Failed to get database test2, returning NoSuchObjectException
Time taken: 0.052 seconds
spark-sql> desc database test;
Database Name	test
Comment
Location	file:/Users/kentyao/Downloads/spark/spark-3.2.0-SNAPSHOT-bin-20210226/datalake/test.db
Owner	kentyao
Time taken: 0.023 seconds, Fetched 4 row(s)
{code}

Another thing is that the log becomes nubilous, for example.

{code:java}
21/02/27 13:54:17 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('datalake').
21/02/27 13:54:17 INFO SharedState: Warehouse path is 'datalake'.
{code}
Bug: Spark-hive: apache hive dependency with CVEs
Status: Open | Priority: Major | Resolution: 
Created: 17/Feb/21 20:06
Affects Version/s: 3.0.1

Apache hive version 2.3.7 used by spark-hive (version 3.0.1) has the following CVEs, as reported by our security team.

CVE-2017-12625, CVE-2015-1772, CVE-2016-3083, CVE-2018-11777, CVE-2014-0228

Please upgrade apache hive libraries to a higher version with no known security risks.
Bug: Wrong discovered partition value
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Feb/21 19:29
Affects Version/s: 2.4.8

The example below portraits the issue:
{code:scala}
      val df = Seq((0, "AA"), (1, "-0")).toDF("id", "part")
      df.write
        .partitionBy("part")
        .format("parquet")
        .save(path)
      val readback = spark.read.parquet(path)
      readback.printSchema()
      readback.show(false)
{code}

It write the partition value as string:
{code}
/private/var/folders/p3/dfs6mf655d7fnjrsjvldh0tc0000gn/T/spark-e09eae99-7ecf-4ab2-b99b-f63f8dea658d
├── _SUCCESS
├── part=-0
│   └── part-00001-02144398-2896-4d21-9628-a8743d098cb4.c000.snappy.parquet
└── part=AA
    └── part-00000-02144398-2896-4d21-9628-a8743d098cb4.c000.snappy.parquet
{code}
*"-0"* and "AA".

but when Spark reads data back, it transforms "-0" to "0"
{code}
root
 |-- id: integer (nullable = true)
 |-- part: string (nullable = true)

+---+----+
|id |part|
+---+----+
|0  |AA  |
|1  |0   |
+---+----+
{code}
Bug: SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/Feb/21 10:00
Affects Version/s: 3.2.0

SPARK-33084 introduced support for Ivy paths in {{sc.addJar}} or Spark SQL {{ADD JAR}}. If we use a custom ivySettings file using {{spark.jars.ivySettings}}, it is loaded at [https://github.com/apache/spark/blob/b26e7b510bbaee63c4095ab47e75ff2a70e377d7/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L1280.] However, this file is only accessible on the client machine. In cluster mode, this file is not available on the driver and so {{addJar}} fails.

{code:sh}
spark-submit --master yarn --deploy-mode cluster --class IvyAddJarExample --conf spark.jars.ivySettings=/path/to/ivySettings.xml example.jar
{code}

{code}
java.lang.IllegalArgumentException: requirement failed: Ivy settings file /path/to/ivySettings.xml does not exist
	at scala.Predef$.require(Predef.scala:281)
	at org.apache.spark.deploy.SparkSubmitUtils$.loadIvySettings(SparkSubmit.scala:1331)
	at org.apache.spark.util.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:176)
	at org.apache.spark.util.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:156)
	at org.apache.spark.sql.internal.SessionResourceLoader.resolveJars(SessionState.scala:166)
	at org.apache.spark.sql.hive.HiveSessionResourceLoader.addJar(HiveSessionStateBuilder.scala:133)
	at org.apache.spark.sql.execution.command.AddJarCommand.run(resources.scala:40)
 {code}

We should ship the ivySettings file to the driver so that {{addJar}} is able to find it.
Bug: Report metrics from Datasource v2 scan
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 03/Feb/21 04:14
Affects Version/s: 3.2.0

This is related to SPARK-34297.

In SPARK-34297, we want to add a couple of useful metrics when reading from Kafka in SS. We need some public API change in DS v2 to make it possible.
Bug: Leverage parquet bloom filters
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Feb/21 18:00
Affects Version/s: 3.2.0

The currently in-progress SPARK-34542 brings in parquet 1.12, which contains PARQUET-41.

From searching the issues, it seems there is no current tracker for this, though I found a [comment|https://issues.apache.org/jira/browse/SPARK-20901?focusedCommentId=17052473&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17052473] from [~dongjoon] that points out the missing parquet support up until now.
Bug: Binary logistic regression incorrectly computes the intercept and coefficients when data is not centered
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/Feb/21 15:12
Affects Version/s: 2.4.5

I have written up a fairly detailed gist that includes code to reproduce the bug, as well as the output of the code and some commentary:
[https://gist.github.com/ykerzhner/51358780a6a4cc33266515f17bf98a96]
To summarize: under certain conditions, the minimization that fits a binary logistic regression contains a bug that pulls the intercept value towards the log(odds) of the target data.  This is mathematically only correct when the data comes from distributions with zero means.  In general, this gives incorrect intercept values, and consequently incorrect coefficients as well.
As I am not so familiar with the spark code base, I have not been able to find this bug within the spark code itself.  A hint to this bug is here: 
[https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala#L894-L904]
based on the code, I don't believe that the features have zero means at this point, and so this heuristic is incorrect.  But an incorrect starting point does not explain this bug.  The minimizer should drift to the correct place.  I was not able to find the code of the actual objective function that is being minimized.
Bug: Allow several properties files
Status: Open | Priority: Major | Resolution: 
Created: 03/Feb/21 13:53
Affects Version/s: 3.0.1

Example: we have 2 applications A and B. These applications have some common Spark settings and some application-specific settings. The idea is to run them like this:
{code:bash}
spark-submit --properties-files common.properties,a.properties A
spark-submit --properties-files common.properties,b.properties B
{code}
Benefits:
 - Common settings can be extracted to a common file _common.properties_, no need to copy them over _a.properties_ and _b.properties_
 - Applications can override common settings in their respective custom properties files

Currently the following mechanism works in SparkSubmitArguments.scala: console arguments like _--conf key=value_ overwrite settings in the properties file. This is not enough, because console arguments should be specified in the launcher script; de-facto they belong to the binary distribution rather than the configuration.

Consider the following scenario: Spark on Kubernetes, the configuration is provided as a ConfigMap. We could have the following ConfigMaps:
 - _a.properties_ // mount to the Pod with application A
 - _b.properties_ // mount to the Pod with application B
 - _common.properties_ // mount to both Pods with A and B
 Meanwhile the launcher script _app-submit.sh_ is the same for both applications A and B, since it contains none configuration settings:

{code:bash}
spark-submit --properties-files common.properties,${app_name}.properties ...
{code}
*Alternate solution*

Use Typesafe Config for Spark settings instead of properties files. Typesafe Config allows including files.
 For example, settings for the application A - _a.conf_:
{code:yaml}
include required("common.conf")

spark.sql.shuffle.partitions = 240
{code}
Bug: Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 21/Feb/21 03:59
Affects Version/s: 3.0.2

For a specific stage, it is useful to show the task metrics in percentile distribution.  This information can help users know whether or not there is a skew/bottleneck among tasks in a given stage.  We list an example in [^taskMetricsDistributions.json]

Similarly, it is useful to show the executor metrics in percentile distribution for a specific stage. This information can show whether or not there is a skewed load on some executors.  We list an example in [^executorMetricsDistributions.json]

 

We define withSummaries query parameter in the REST API for a specific stage as:

applications/<application_id>/<application_attempt/stages/<stage_id>/<stage_attempt>?withSummaries=[true|false]&quantiles=0.0,0.25,0.5,0.75,1.0
 * When withSummaries=true, both task metrics in percentile distribution and executor metrics in percentile distribution are included in the REST API output.  The default value of withSummaries is false, i.e. no metrics percentile distribution will be included in the REST API output.
 * Query parameter quantiles define the quantiles we use to calculate metrics distributions.  It takes effect only when {{withSummaries=true.}}  Its default value is {{0.0,0.25,0.5,0.75,1.0.}}
Bug: java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.<init>
Status: In Progress | Priority: Major | Resolution: 
Created: 28/Feb/21 13:41
Affects Version/s: 3.0.1

Using Apache Spark 3.0 with Scala 2.12, and trying to submit a job, which reads from Azure Event Hub as binary encoded and then deserializes as protobuf generated class, then getting this error. The stack trace is follows:
{code:java}
java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.<init>(Lorg/apache/spark/sql/types/StructType;ZLscala/collection/Seq;Lorg/apache/spark/sql/catalyst/expressions/Expression;Lscala/reflect/ClassTag;)V
	at frameless.TypedExpressionEncoder$.apply(TypedExpressionEncoder.scala:45)
	at scalapb.spark.Implicits.typedEncoderToEncoder(TypedEncoders.scala:125)
	at scalapb.spark.Implicits.typedEncoderToEncoder$(TypedEncoders.scala:122)
	at scalapb.spark.Implicits$.typedEncoderToEncoder(TypedEncoders.scala:128)
	at Utils.MessagingQueues.EventHubSourceReader$.editorialAdStream(EventHubSourceReader.scala:57)
	at Utils.MessagingQueues.SourceReader$.readEditorialAdSource(SourceReader.scala:39)
	at Workflows.Streaming.PROD.UnifiedNRT.UnifiedNRTHelper$.fetchInputStream(UnifiedNRTHelper.scala:62)
	at Workflows.Streaming.PROD.UnifiedNRT.UnifiedNRT$.main(UnifiedNRT.scala:50)
	at Workflows.Streaming.PROD.UnifiedNRT.UnifiedNRT.main(UnifiedNRT.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:775)


{code}
Bug: Optimize WAL commit phase on SS
Status: Closed | Priority: Major | Resolution: Fixed
Created: 06/Feb/21 04:02
Affects Version/s: 3.2.0

I found there're unnecessary access / expensive operation of file system in WAL commit phase of SS.

They can be optimized via caching (using driver memory a bit) & replacing FS operation. This brings reduced latency per batch, especially checkpoint against object store.
Bug: Avoid unnecessary view resolving and remove the `performCheck` flag
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 23/Feb/21 05:51
Affects Version/s: 3.1.1

in SPARK-34490, I added a `performCheck` flag to skip analysis check when resolving views. This is due to some view resolution is unnecessary. So we can avoid these unnecessary view resolution and remove the `performCheck` flag.
Bug: breeze.linalg.SparseVector.equals always return true may bring error.
Status: Open | Priority: Major | Resolution: 
Created: 24/Feb/21 08:48
Affects Version/s: 3.0.2

I implemented an iterative graph algorithm with 'breeze.linalg.SparseVector' just like 'Graph.staticParallelPersonalizedPageRank' does. However, the algorithm can not iterate correctly and I finally found that is because 'breeze.linalg.SparseVector.equals' will always return 'true' if two vectors have the same dimension.

 
{code:java}
scala> val zero =  SparseVector.zeros[Double](10)
zero: breeze.linalg.SparseVector[Double] = SparseVector(10)()
scala> val vec = SparseVector(10)((0, 1.0))
vec: breeze.linalg.SparseVector[Double] = SparseVector(10)((0,1.0))
scala> zero == vec
res2: Boolean = true
{code}
 

Does that bring the same error for 'Graph.staticParallelPersonalizedPageRank' ?
Bug: Issue Tracker for JDK related Bugs
Status: Resolved | Priority: Major | Resolution: Done
Created: 24/Feb/21 15:10
Affects Version/s: 2.4.7

This JIRA is used to log JDK-related issues that often cause Spark to throw weird runtime exceptions or be permanently unresponsive.

For Spark users, these issues tend to be very common but difficult to locate. When users encounter such questions, the JIRA may help them get quick answers when googling. The answers may often simply require them to upgrade the JDK versions.

These issues are also difficult for the community to deal with in Spark's code, and even maintaining documentation for troubleshooting in the code can be a challenge. As a distributed JVM application, problems with the JDK can take many forms.

So JIRA might be a good choice for documenting these problems
Bug: DataSource V2: Add default null ordering to SortDirection
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Feb/21 19:14
Affects Version/s: 3.2.0

The SQL standard does not define the default null ordering for a sort direction. That's why it is up to a query engine. We need to standardize this in our public connector expressions to avoid ambiguity. That's why I propose to match the Catalyst expressions.
Bug: Make dynamic allocation upscaling more progressive on K8S
Status: In Progress | Priority: Major | Resolution: 
Created: 23/Feb/21 16:09
Affects Version/s: 2.3.4

Currently even a single late pod request stops upscaling. As we have allocation batch size it would be better to go up to that limit as soon as possible (if serving pod requests are slow we have to make them as early as possible).
Bug: Correct the active SparkSession for streaming query
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 20/Feb/21 08:38
Affects Version/s: 2.4.7

[https://github.com/apache/spark/blob/6b34745cb9b294c91cd126c2ea44c039ee83cb84/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala#L326]

 

It should be `sparkSessionForStream`.
Bug: CreateTableAsSelect should have metrics update too
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 28/Feb/21 06:09
Affects Version/s: 3.1.1

For command `CreateTableAsSelect` we use `InsertIntoHiveTable`, `InsertIntoHadoopFsRelationCommand` to insert data.
We will update metrics of `InsertIntoHiveTable`, `InsertIntoHadoopFsRelationCommand` in `FileFormatWriter.write()`, but we only show CreateTableAsSelectCommand in WebUI SQL Tab.
We need to update `CreateTableAsSelectCommand`'s metrics too.
Bug: Dynamic allocation on K8s kills executors with running tasks
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/Feb/21 16:06
Affects Version/s: 3.0.0

There is race between executor POD allocator and cluster scheduler backend. 
During downscaling (in dynamic allocation) we experienced a lot of killed new executors with running task on them.

The pattern in the log is the following:

{noformat}
21/02/01 15:12:03 INFO ExecutorMonitor: New executor 312 has registered (new total is 138)
...
21/02/01 15:12:03 INFO TaskSetManager: Starting task 247.0 in stage 4.0 (TID 2079, 100.100.18.138, executor 312, partition 247, PROCESS_LOCAL, 8777 bytes)
21/02/01 15:12:03 INFO ExecutorPodsAllocator: Deleting 3 excess pod requests (408,312,307).
...
21/02/01 15:12:04 ERROR TaskSchedulerImpl: Lost executor 312 on 100.100.18.138: The executor with id 312 was deleted by a user or the framework.
{noformat}
Bug: DPP support LIKE ANY/ALL
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Feb/21 13:10
Affects Version/s: 3.2.0

Support this case:
{code:sql}
SELECT date_id, product_id FROM fact_sk f
JOIN dim_store s
ON f.store_id = s.store_id WHERE s.country LIKE ANY ('%D%E%', '%A%B%')
{code}