av9ash/GitBugs huggingface dataset:
Bug: Casting invalid string to day-time interval should return null
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 16/Aug/21 13:45
Affects Version/s: 3.2.0

Currently when casting invalid string to day-time interval, Spark always throws an exception, with/without ANSI mode enabled.
This should only happen when ANSI mode is on. When ANSI mode is off, Spark should return null for invalid cast.
Bug: Casting invalid string to year-month interval should return null
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 16/Aug/21 13:33
Affects Version/s: 3.2.0

Currently when casting invalid string to year-month interval, Spark always throws an exception, with/without ANSI mode enabled.
This is inconsistent with the behavior of other type 
cast. When ANSI mode is off, Spark should return null for invalid cast.
Bug: Make from_json/to_json to handle timestamp_ntz type properly
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/Aug/21 07:29
Affects Version/s: 3.3.0

In the current master, to_json/from_json can handle timestamp type like as follows.
{code:java}
SELECT to_json(map("a", TIMESTAMP"2021-11-23 11:22:33"));
{"a":"2021-11-23T11:22:33.000+09:00"}
{code}
{code:java}
SELECT from_json('{"a":"2021-11-23 11:22:33"}', "a TIMESTAMP");
{"a":2021-11-23 11:22:33}
{code}
But they cannot handle timestamp_ntz type properly.
{code:java}
SELECT to_json(map("a", TIMESTAMP_NTZ"2021-11-23 11:22:33"));
21/08/12 16:14:07 ERROR SparkSQLDriver: Failed in [SELECT to_json(map("a", TIMESTAMP_NTZ"2021-11-23 11:22:33"))]
java.lang.RuntimeException: Failed to convert value 1637666553000000 (class of class java.lang.Long) with the type of TimestampNTZType to JSON.
        at org.apache.spark.sql.errors.QueryExecutionErrors$.failToConvertValueToJsonError(QueryExecutionErrors.scala:294)
        at org.apache.spark.sql.catalyst.json.JacksonGenerator.$anonfun$makeWriter$25(JacksonGenerator.scala:201)
        at org.apache.spark.sql.catalyst.json.JacksonGenerator.$anonfun$makeWriter$25$adapted(JacksonGenerator.scala:199)
        at org.apache.spark.sql.catalyst.json.JacksonGenerator.writeMapData(JacksonGenerator.scala:253)
        at org.apache.spark.sql.catalyst.json.JacksonGenerator.$anonfun$write$3(JacksonGenerator.scala:293)
        at org.apache.spark.sql.catalyst.json.JacksonGenerator.writeObject(JacksonGenerator.scala:206)
        at org.apache.spark.sql.catalyst.json.JacksonGenerator.write(JacksonGenerator.scala:292)
{code}
{code:java}
SELECT from_json('{"a":"2021-11-23 11:22:33"}', "a TIMESTAMP_NTZ");
21/08/12 16:16:00 ERROR SparkSQLDriver: Failed in [SELECT from_json('{"a":"2021-11-23 11:22:33"}', "a TIMESTAMP_NTZ")]
java.lang.Exception: Unsupported type: timestamp_ntz
        at org.apache.spark.sql.errors.QueryExecutionErrors$.unsupportedTypeError(QueryExecutionErrors.scala:777)
        at org.apache.spark.sql.catalyst.json.JacksonParser.makeConverter(JacksonParser.scala:339)
        at org.apache.spark.sql.catalyst.json.JacksonParser.$anonfun$makeConverter$17(JacksonParser.scala:313)
{code}
Bug: Disallow binary operations between Interval and String literal
Status: Resolved | Priority: Major | Resolution: Done
Created: 13/Aug/21 09:31
Affects Version/s: 3.2.0

If a binary operation contains interval type and string literal, we can't decide which interval type the string literal should be promoted as. There are many possible interval types, such as year interval, month interval, day interval, hour interval, etc.
The related binary operation for Interval should be 
* Add
* Subtract
* Comparisions
Bug: Make from_csv/to_csv to handle timestamp_ntz type properly
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/Aug/21 07:25
Affects Version/s: 3.3.0

In the current master, to_csv/from_csv can handle timestamp type like as follows.
{code}
SELECT to_csv(struct(TIMESTAMP"2021-11-23 11:22:33")); 
2021-11-23T11:22:33.000+09:00

SELECT from_csv("2021-11-23 11:22:33", "a TIMESTAMP"); 
{"a":2021-11-23 11:22:33}
{code}

But they cannot handle timestamp_ntz type properly.
{code}
SELECT to_csv(struct(TIMESTAMP_NTZ"2021-11-23 11:22:33"));
-- 2021-11-23T11:22:33.000 is expected.
1637666553000000

SELECT from_csv("2021-11-23 11:22:33", "a TIMESTAMP_NTZ");
21/08/12 16:12:49 ERROR SparkSQLDriver: Failed in [SELECT from_csv("2021-11-23 11:22:33", "a TIMESTAMP_NTZ")]
java.lang.Exception: Unsupported type: timestamp_ntz
        at org.apache.spark.sql.errors.QueryExecutionErrors$.unsupportedTypeError(QueryExecutionErrors.scala:777)
        at org.apache.spark.sql.catalyst.csv.UnivocityParser.makeConverter(UnivocityParser.scala:234)
        at org.apache.spark.sql.catalyst.csv.UnivocityParser.$anonfun$valueConverters$1(UnivocityParser.scala:134)
{code}
Bug: Support Interval add/subtract NULL
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/Aug/21 14:33
Affects Version/s: 3.2.0

Currently, `null + interval`  will become `cast(cast(null as timestamp) + interval) as null`. This is a unexpected behavior and the result should not be of null type.
This weird behavior applies to `null - interval`, `interval + null`, `interval - null` as well.
To change it, I propose to cast the null as the same data type of the other element in the add/subtract:
```
null + interval => cast(null as interval) + interval
null - interval => cast(null as interval) - interval
interval + null=> interval + cast(null as interval)
interval - null => interval - cast(null as interval)
```
Bug: Fix intermittent test failure due to netty dependency version bump
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 11/Aug/21 20:53
Affects Version/s: 3.2.0

In SPARK-35132, Spark's netty dependency version was bumped from 4.1.51 to 4.1.63.

Since Netty version 4.1.52, a Netty specific io.netty.channel.StacklessClosedChannelException gets thrown when Netty's AbstractChannel encounters a closed channel.

This can sometimes break the test org.apache.spark.network.RPCIntegrationSuite as reported [here|[https://github.com/apache/spark/pull/33613#issuecomment-896697401].]

This is due to the hardcoded list of exception messages to check in RPCIntegrationSuite does not include this new StacklessClosedChannelException.
Bug: SortMergeJoin do unnecessary shuffle for tables whose provider is hive
Status: In Progress | Priority: Major | Resolution: 
Created: 12/Aug/21 10:49
Affects Version/s: 3.1.1

Create two tables in spark like this:
{code:java}
CREATE TABLE catalog_sales
(
    cs_sold_date_sk           BIGINT,
    cs_sold_time_sk           BIGINT,
    cs_ship_date_sk           BIGINT,
    cs_bill_customer_sk       BIGINT,
    cs_bill_cdemo_sk          BIGINT,
    cs_bill_hdemo_sk          BIGINT,
    cs_bill_addr_sk           BIGINT,
    cs_ship_customer_sk       BIGINT,
    cs_ship_cdemo_sk          BIGINT,
    cs_ship_hdemo_sk          BIGINT,
    cs_ship_addr_sk           BIGINT,
    cs_call_center_sk         BIGINT,
    cs_catalog_page_sk        BIGINT,
    cs_ship_mode_sk           BIGINT,
    cs_warehouse_sk           BIGINT,
    cs_item_sk                BIGINT,
    cs_promo_sk               BIGINT,
    cs_order_number           BIGINT,
    cs_quantity               BIGINT,
    cs_wholesale_cost         FLOAT,
    cs_list_price             FLOAT,
    cs_sales_price            FLOAT,
    cs_ext_discount_amt       FLOAT,
    cs_ext_sales_price        FLOAT,
    cs_ext_wholesale_cost     FLOAT,
    cs_ext_list_price         FLOAT,
    cs_ext_tax                FLOAT,
    cs_coupon_amt             FLOAT,
    cs_ext_ship_cost          FLOAT,
    cs_net_paid               FLOAT,
    cs_net_paid_inc_tax       FLOAT,
    cs_net_paid_inc_ship      FLOAT,
    cs_net_paid_inc_ship_tax  FLOAT,
    cs_net_profit             FLOAT
)
clustered by (cs_item_sk)  into  10 buckets
stored as ORC;

CREATE TABLE store_sales
(
    ss_sold_date_sk           BIGINT,
    ss_sold_time_sk           BIGINT,
    ss_item_sk                BIGINT,
    ss_customer_sk            BIGINT,
    ss_cdemo_sk               BIGINT,
    ss_hdemo_sk               BIGINT,
    ss_addr_sk                BIGINT,
    ss_store_sk               BIGINT,
    ss_promo_sk               BIGINT,
    ss_ticket_number          BIGINT,
    ss_quantity               BIGINT,
    ss_wholesale_cost         FLOAT,
    ss_list_price             FLOAT,
    ss_sales_price            FLOAT,
    ss_ext_discount_amt       FLOAT,
    ss_ext_sales_price        FLOAT,
    ss_ext_wholesale_cost     FLOAT,
    ss_ext_list_price         FLOAT,
    ss_ext_tax                FLOAT,
    ss_coupon_amt             FLOAT,
    ss_net_paid               FLOAT,
    ss_net_paid_inc_tax       FLOAT,
    ss_net_profit             FLOAT
)
CLUSTERED BY (ss_item_sk) INTO 10 buckets
stored as ORC;{code}
Then the table's provider is hive, when execute sql:
{code:java}
SELECT * FROM tpcds_orc_mintest.store_sales a join tpcds_orc_mintest.catalog_sales b on a.ss_item_sk = b.cs_item_sk limit 1000;
{code}
Spark do unnecessary shuffle before SortMergeJoin. I found that when create LogicalRelation for the hive table in HiveMetastoreCatalog.covertToLogicalRelation, the bucketSpec is missing.
Bug: ANSI type coercion rule for date time operations
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Aug/21 09:10
Affects Version/s: 3.2.0

Currently the type coercion rule `DateTimeOperations` doesn't match the design of the ANSI type coercion system:
1. For date_add/date_sub, if the input is timestamp type, Spark should not convert it into date type since date type is narrower than timestamp.
2. For date_add/date_sub/time_add, string value can be implicit cast to date/timestamp only when it is literal.

Thus, we need to have a new rule for the date time operations in ANSI type coercion system.
Bug: Aggregate functions over no grouping keys, on tables with a single bucket, return multiple rows
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/Aug/21 07:08
Affects Version/s: 3.1.0

When running any aggregate function, without any grouping keys, on a table with a single bucket, multiple rows are returned. 

This happens because the aggregate function satisfies the `AllTuples` distribution, no `Exchange` will be planned, and the bucketed scan will be disabled.

 

Reproduction:

 
{code:java}
sql(
   """
   |CREATE TABLE t1 (`id` BIGINT, `event_date` DATE)
   |USING PARQUET
   |CLUSTERED BY (id)
   |INTO 1 BUCKETS
   |""".stripMargin)

sql(
   """
   |INSERT INTO TABLE t1 VALUES(1.23, cast("2021-07-07" as date))
   |""".stripMargin)

sql(
   """
   |INSERT INTO TABLE t1 VALUES(2.28, cast("2021-08-08" as date))
   |""".stripMargin)

assert(sql("select sum(id) from t1 where id is not null").count == 1){code}
Bug: Minor changes to address a few identified server side inefficiencies
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Aug/21 04:39
Affects Version/s: 3.2.0

With the SPIP ticket close to being finished, we have done some performance evaluations to compare the performance of push-based shuffle in upstream Spark with the production version we have internally at LinkedIn.

The evaluations have revealed a few regressions and also some additional perf improvement opportunity.
Bug: Support comparison of ANSI intervals with different fields
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 05/Aug/21 17:54
Affects Version/s: 3.2.0

Support comparison of
- a day-time interval with another day-time interval which has different fields
- a year-month interval with another year-month interval where fields are different.

The example below shows the issue:

{code:sql}
spark-sql> select interval '1' day > interval '1' hour;
Error in query: cannot resolve '(INTERVAL '1' DAY > INTERVAL '01' HOUR)' due to data type mismatch: differing types in '(INTERVAL '1' DAY > INTERVAL '01' HOUR)' (interval day and interval hour).; line 1 pos 7;
'Project [unresolvedalias((INTERVAL '1' DAY > INTERVAL '01' HOUR), None)]
+- OneRowRelation

spark-sql> select interval '2' year > interval '11' month;
Error in query: cannot resolve '(INTERVAL '2' YEAR > INTERVAL '11' MONTH)' due to data type mismatch: differing types in '(INTERVAL '2' YEAR > INTERVAL '11' MONTH)' (interval year and interval month).; line 1 pos 7;
'Project [unresolvedalias((INTERVAL '2' YEAR > INTERVAL '11' MONTH), None)]
+- OneRowRelation
{code}
Bug: Add the support in Spark for having group by map datatype column for the scenario that works in Hive
Status: In Progress | Priority: Major | Resolution: 
Created: 08/Aug/21 13:11
Affects Version/s: 3.0.3

Add the support in Spark for having group by map datatype column for the scenario that works in Hive.

In hive the below scenario works 
{code:java}
describe extended complex2;
OK
id                  string 
c1                  map<int, string>   
Detailed Table Information Table(tableName:complex2, dbName:default, owner:abc, createTime:1627994412, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:id, type:string, comment:null), FieldSchema(name:c1, type:map<int,string>, comment:null)], location:/user/hive/warehouse/complex2, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1})

select * from complex2;
OK
1 {1:"u"}
2 {1:"u",2:"uo"}
1 {1:"u",2:"uo"}
Time taken: 0.363 seconds, Fetched: 3 row(s)

Working Scenario in Hive -: 

select id, c1, count(*) from complex2 group by id, c1;
OK
1 {1:"u"} 1
1 {1:"u",2:"uo"} 1
2 {1:"u",2:"uo"} 1
Time taken: 1.621 seconds, Fetched: 3 row(s)

Failed Scenario in Hive -: failed when map type is present in aggregated expression 

select id, max(c1), count(*) from complex2 group by id, c1;

FAILED: UDFArgumentTypeException Cannot support comparison of map<> type or complex type containing map<>.
{code}
But in spark this scenario where the group by map column failed for this scenario where the map column is used in the select without any aggregation
{code:java}
scala> spark.sql("select id,c1, count(*) from complex2 group by id, c1").show
org.apache.spark.sql.AnalysisException: expression spark_catalog.default.complex2.`c1` cannot be used as a grouping expression because its data type map<int,string> is not an orderable data type.;
Aggregate [id#1, c1#2], [id#1, c1#2, count(1) AS count(1)#3L]
+- SubqueryAlias spark_catalog.default.complex2
 +- HiveTableRelation [`default`.`complex2`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [id#1, c1#2], Partition Cols: []]
at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis(CheckAnalysis.scala:50)
{code}
There is need to add the this scenario where grouping expression can have map type if aggregated expression does not have the that map type reference. This helps in migrating the user from hive to Spark.

After the code change 
{code:java}
scala> spark.sql("select id,c1, count(*) from complex2 group by id, c1").show
+---+-----------------+--------+                                                
| id|               c1|count(1)|
+---+-----------------+--------+
|  1|         {1 -> u}|       1|
|  2|{1 -> u, 2 -> uo}|       1|
|  1|{1 -> u, 2 -> uo}|       1|
+---+-----------------+--------+
 {code}
Bug: Spark3 fails to read hive table with mixed format
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 05/Aug/21 23:20
Affects Version/s: 3.0.0

Spark3 fails to read hive table with mixed format with hive Serde, this is a regression compares to Spark 2.4. 

Replication steps :
 1. In spark 3 (3.0 or 3.1) spark shell:
{code:java}
scala> spark.sql("create table tmp.test_table (id int, name string) partitioned by (pt int) stored as rcfile")

scala> spark.sql("insert into tmp.test_table (pt = 1) values (1, 'Alice'), (2, 'Bob')")
{code}
2. Run hive command to change table file format (from RCFile to Parquet).
{code:java}
hive (default)> alter table set tmp.test_table fileformat Parquet;
{code}
3. Try to read partition (in RCFile format) with hive serde using Spark shell:
{code:java}
scala> spark.conf.set("spark.sql.hive.convertMetastoreParquet", "false")

scala> spark.sql("select * from tmp.test_table where pt=1").show{code}
Exception: (anonymized file path with <path>)
{code:java}
Caused by: java.lang.RuntimeException: s3a://<path>/data/part-00000-22112178-5dd7-4065-89d7-2ee550296909-c000 is not a Parquet file. expected magic number at tail [80, 65, 82, 49] but found [5, 96, 1, -33]
  at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:524)
  at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:505)
  at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:499)
  at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:448)
  at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:433)
  at org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:79)
  at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:75)
  at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:60)
  at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:75)
  at org.apache.spark.rdd.HadoopRDD$$anon$1.liftedTree1$1(HadoopRDD.scala:286)
  at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:285)
  at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:243)
  at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:96)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  <omitted more logs>{code}
Bug: Randomize blocks within a push request before pushing to improve block merge ratio
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 05/Aug/21 06:18
Affects Version/s: 3.2.0

On the client side, we are currently randomizing the order of push requests before processing each request. In addition we can further randomize the order of blocks within each push request before pushing them.

In our benchmark, this has resulted in a 60%-70% reduction of blocks that fail to be merged due to bock collision (the existing block merge ratio is already pretty good in general, and this further improves it).
Bug: Scala API: support CrossValidatorModel get standard deviation of metrics for each paramMap
Status: Open | Priority: Major | Resolution: 
Created: 05/Aug/21 09:25
Affects Version/s: 3.2.0

This is the parity feature of https://issues.apache.org/jira/browse/SPARK-36425

Note:
We need also update PySpark CrossValidatorModel.to_java/from_java methods in this task.

Note2:
We need support loading old version saved CrossValidatorModel
Bug: Pin pyzmq to 2.22.0
Status: Resolved | Priority: Major | Resolution: Invalid
Created: 05/Aug/21 09:23
Affects Version/s: 3.2.0

GA is failing and the root cause seems the latest release of pyzmq, which is released a few hours ago.
https://github.com/apache/spark/runs/3250261989#step:11:414
https://github.com/apache/spark/runs/3250252645?check_suite_focus=true#step:11:417
https://pypi.org/project/pyzmq/
Bug: Add SQL metrics to AdaptiveSparkPlanExec for BHJs and Skew joins
Status: In Progress | Priority: Major | Resolution: 
Created: 04/Aug/21 17:14
Affects Version/s: 3.1.2

Add {{"num broadcast joins conversions"}} and {{"num skew join conversions"}}
 metrics to {{AdaptiveSparkPlanExec}} so that it's easy to get a sense of how much impact AQE had on an a complex query.

It's also useful for systems that collect metrics for later analysis of AQE effectiveness in large production deployment.
Bug: DecimalPrecision only add necessary cast
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Aug/21 12:32
Affects Version/s: 3.3.0

For example:
{noformat}
EqualTo(AttributeReference("d1", DecimalType(5, 2))(), AttributeReference("d2", DecimalType(2, 1))())
{noformat}
It will add a useless cast to {{d1}}:
{noformat}
(cast(d1#6 as decimal(5,2)) = cast(d2#7 as decimal(5,2)))
{noformat}
Bug: Add possibility for jdbc insert hints
Status: Open | Priority: Major | Resolution: 
Created: 02/Aug/21 16:36
Affects Version/s: 2.4.7

Some SQL backends supports hints for SQL insert statement, such as
{code:java}
/*+ ignore_row_on_dupkey_index ( table(col1, col2, ...) ) */ {code}
for example.
{code:java}
insert  /*+ ignore_row_on_dupkey_index ( table(col1, col2, ...) ) */ into table(...{code}
But spark jdbc writer does not allow to add hints.

I suggest to add support for hints in org.apache.spark.sql.execution.datasources.jdbc.JdbsUtils.getInsertStatement and org.apache.spark.sql.execution.datasources.jdbc.JdbsUtils.getInsertStatement.saveTable

Hints should be stored in options.

I already have version with hints support so if issue will be accepted I can post the fix.
Bug: Removes outer join if all grouping and aggregate expressions are from the streamed side
Status: In Progress | Priority: Minor | Resolution: 
Created: 11/Aug/21 10:45
Affects Version/s: 3.2.0

Removes outer join if all grouping and aggregate expressions are from the streamed side.

For example:
{code:java}
spark.range(200L).selectExpr("id AS a", "id as b", "id as c").createTempView("t1")
spark.range(300L).selectExpr("id AS a").createTempView("t2")
spark.sql("SELECT t1.b, max(t1.c) as c FROM t1 LEFT JOIN t2 ON t1.a = t2.a GROUP BY t1.b").explain(true)
{code}
Current optimized plan:
{code:java}
== Optimized Logical Plan ==
Aggregate [b#3L], [b#3L, max(c#4L) AS c#20L]
+- Project [b#3L, c#4L]
   +- Join LeftOuter, (a#2L = a#10L)
      :- Project [id#0L AS a#2L, id#0L AS b#3L, id#0L AS c#4L]
      :  +- Range (0, 200, step=1, splits=Some(1))
      +- Project [id#8L AS a#10L]
         +- Range (0, 300, step=1, splits=Some(1))
{code}
Expected optimized plan:
{code:java}
== Optimized Logical Plan ==
Aggregate [b#277L], [b#277L, max(c#278L) AS c#290L]
+- Project [id#274L AS b#277L, id#274L AS c#278L]
   +- Range (0, 200, step=1, splits=Some(2))
{code}
Bug: Replace anonymous classes with lambda expressions
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 04/Aug/21 08:07
Affects Version/s: 3.3.0

Anonymous classes can be replaced with lambda expressions in Java code, for example

*before*
{code:java}
new Thread(new Runnable() {
    @Override
    public void run() {
      // run thread
    }
  });
{code}
*after*
{code:java}
new Thread(() -> {
    // run thread
  });
{code}
Bug: Upgrade Jetty version to 9.4.43
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 05/Aug/21 20:33
Affects Version/s: 3.2.0

Upgrade Jetty version to 9.4.43.v20210629 in current Spark master in order to fix vulnerability https://nvd.nist.gov/vuln/detail/CVE-2021-34429.