av9ash/GitBugs huggingface dataset:
Bug: Avoid unnecessary `setReplication` in Yarn mode
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Nov/21 07:31
Affects Version/s: 3.1.2

We found a large number of replication logs in hdfs server   
```
2021-11-04,17:22:13,065 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: Replication remains unchanged at 3 for xxx/.sparkStaging/application_1635470728320_1144379/__spark_libs__303253482044663796.zip
2021-11-04,17:22:13,069 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: Replication remains unchanged at 3 for xxx/.sparkStaging/application_1635470728320_1144383/__spark_libs__4747402134564993861.zip
2021-11-04,17:22:13,070 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: Replication remains unchanged at 3 for xxx/.sparkStaging/application_1635470728320_1144373/__spark_libs__4377509773730188331.zip
```
https://github.com/apache/hadoop/blob/6f7b965808f71f44e2617c50d366a6375fdfbbfa/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java#L2439  
`setReplication` needs to acquire write lock, we should reduce this unnecessary operation
Bug: option -v is broken?
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 08/Nov/21 10:38
Affects Version/s: 3.2.0

Hi,

Here is the output:

 

{{$ spark-shell -v}}

{{*[CUT]*}}
{{Parsed arguments:}}
{{*[CUT]*}}
{{Main class:}}
{{org.apache.spark.repl.Main}}
{{Arguments:}}
{{--verbose}}
{{Spark config:}}

{{*[CUT]*}}
{{Classpath elements:}}
{{Setting default log level to "WARN".}}
{{To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).}}
{{bad option: '--verbose'}}
Bug: Cannot read partitioned parquet files with ANSI interval partition values
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Nov/21 07:59
Affects Version/s: 3.3.0

The code below demonstrates the issue:
{code:scala}
scala> sql("SELECT INTERVAL '1' YEAR AS i, 0 as id").write.partitionBy("i").parquet("/Users/maximgekk/tmp/ansi_interval_parquet")


scala> spark.read.schema("i INTERVAL YEAR, id INT").parquet("/Users/maximgekk/tmp/ansi_interval_parquet").show(false)
21/11/08 10:56:36 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 2)
java.lang.RuntimeException: DataType INTERVAL YEAR is not supported in column vectorized reader.
	at org.apache.spark.sql.execution.vectorized.ColumnVectorUtils.populate(ColumnVectorUtils.java:100)
	at org.apache.spark.sql.execution.datasources.parquet.VectorizedParquetRecordReader.initBatch(VectorizedParquetRecordReader.java:243)
{code}
Bug: Add a deterministic field to QueryPlan
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Nov/21 18:08
Affects Version/s: 3.2.0

We have a _deterministic_ field in [Expressions|https://github.com/apache/spark/blob/b78167a2ee6b11b1f2839274e23676411f919115/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala#L115] to check if an expression is deterministic, but we do not have a similar field in [QueryPlan.|https://github.com/apache/spark/blob/b78167a2ee6b11b1f2839274e23676411f919115/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala#L44]

We have a need for such a check in the QueryPlan sometimes, like in [InlineCTE|https://github.com/apache/spark/blob/b78167a2ee6b11b1f2839274e23676411f919115/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/InlineCTE.scala#L56]

This proposal is to add a _deterministic_ field to QueryPlan.

More details [in this document|https://docs.google.com/document/d/1eIiaSJf-Co2HhjsaQxFNGwUxobnHID4ZGmJMcVytREc/edit#].
Bug: Filter push down through window
Status: In Progress | Priority: Major | Resolution: 
Created: 06/Nov/21 15:48
Affects Version/s: 3.3.0

{code:scala}
spark.sql("CREATE TABLE t1 using parquet as select id as a, id as b from range(1000)")
spark.sql("select * from (SELECT a, count(*) cnt, row_number() over (order by a desc) as rn from t1 group by a) where rn <= 10").explain(true)
{code}

We can optimize this query:
{noformat}
== Optimized Logical Plan ==
Filter (rn#4 <= 10)
+- Window [row_number() windowspecdefinition(a#7L DESC NULLS LAST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rn#4], [a#7L DESC NULLS LAST]
   +- GlobalLimit 10
      +- LocalLimit 10
         +- Sort [a#7L DESC NULLS LAST], true
            +- Aggregate [a#7L], [a#7L, count(1) AS cnt#3L]
               +- Project [a#7L]
                  +- Relation default.t1[a#7L,b#8L] parquet
{noformat}
Bug: Support mapping Spark gpu/fpga resource types to custom YARN resource type
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/Nov/21 15:18
Affects Version/s: 3.0.0

Currently Spark supports gpu/fpga resource scheduling and specifically on YARN it knows how to map gpu/fpga to the YARN resource types yarn.io/gpu and yarn.io/fpga.  YARN also supports custom resource types and in Hadoop 3.3.1 made it easier for users to plugin in custom resource types. This means users may create a custom resource type that represents a GPU or FPGAs because they want additional logic that YARN the built in versions don't have. Ideally Spark users still just  use the generic "gpu" or "fpga" types in Spark. So we should add the ability to change the Spark internal mappings.
Bug: Support Application Timeouts on YARN
Status: In Progress | Priority: Major | Resolution: 
Created: 05/Nov/21 09:23
Affects Version/s: 3.3.0

Since #YARN-3813/2.9.0/3.0.0, YARN supports ApplicationTimeouts. It helps enforce lifetime application SLAs. Currently, lifetime indicates the overall time spent by an application in YARN. It is calculated from its submit time to finish time, including running time and the waiting time for resource allocation.

YARN allows admins to set lifetime of an application at leaf-queue. It also allows users to set it programmatically. During application submission, user can set it in  ApplicationSubmissionContext#setApplicationTimeouts(Map<ApplicationTimeoutType, Long> applicationTimeouts).

 

So far, YARN supports for one timeout type - LIFETIME
Bug: pyspark.pandas fails to create a histogram of one column from a large DataFrame
Status: Open | Priority: Major | Resolution: 
Created: 01/Nov/21 21:11
Affects Version/s: 3.2.0

 

When trying to create a histogram from one column of a large DataFrame, pyspark.pandas fails. So this line
{quote}DF.plot.hist(column="FullVaxPer100", bins=20)  # there are many other columns
{quote}
yields this error
{quote}cannot resolve 'least(min(EndDate), min(EndDeaths), min(`STATE-COUNTY`), min(StartDate), min(StartDeaths), min(POPESTIMATE2020), min(ST_ABBR), min(VaxStartDate), min(Series_Complete_Yes_Start), min(Administered_Dose1_Recip_Start), min(VaxEndDate), min(Series_Complete_Yes_End), min(Administered_Dose1_Recip_End), min(Deaths), min(Series_Complete_Yes_Mid), min(Administered_Dose1_Recip_Mid), min(FullVaxPer100), min(OnePlusVaxPer100), min(DeathsPer100k))' due to data type mismatch: The expressions should all have the same type, got LEAST(timestamp, bigint, string, timestamp, bigint, bigint, string, timestamp, bigint, bigint, timestamp, bigint, bigint, bigint, double, double, double, double, double).;
{quote}
The odd thing is that pyspark.pandas seems to be operating on all the columns when only one is needed.

As a workaround, you can first create a one-column DataFrame that selects just the field you want, then make a histogram of that. But the command above should work also.

I can supply the complete program and datasets that demonstrate the error.
Bug: pyspark.pandas should support tseries.offsets
Status: Open | Priority: Major | Resolution: 
Created: 01/Nov/21 21:02
Affects Version/s: 3.2.0

In regular pandas you can use pandas.offsets to create a time delta. This allows a line like
{quote}this_period_start = OVERALL_START_DATE + pd.offsets.Day(NN)
{quote}
But this does not work in pyspark.pandas.

There are good workarounds, such as datetime.timedelta(days=NN), but pandas programmers would like to move code to pyspark without changing it.
Bug: pyspark.pandas.DataFrame.map() should support .fillna()
Status: Open | Priority: Major | Resolution: 
Created: 01/Nov/21 18:59
Affects Version/s: 3.2.0

In regular pandas you can say 
{quote}DF["new_column"] = DF["column"].map(some_map).fillna(DF["column"])
{quote}
In order to use the existing value if the mapping key is not found.

But this does not work in pyspark.pandas.
Bug:  pyspark.pandas should support DF["column"].str.split("some_suffix").str[0]
Status: Open | Priority: Major | Resolution: 
Created: 01/Nov/21 19:11
Affects Version/s: 3.2.0

In regular pandas you can say
{quote}DF["column"] = DF["column"].str.split("suffix").str[0]
{quote}
In order to strip off a suffix.

With pyspark.pandas, this syntax does not work. You have to say something like
{quote}DF["column"] = DF["column"].str.replace("suffix", '', 1)
{quote}
which works fine if the suffix only appears once at the end, but is not really the same.
Bug: ANSI mode: Add a config to allow casting between Datetime and Numeric
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Nov/21 13:48
Affects Version/s: 3.3.0

Add a config `spark.sql.ansi.allowCastBetweenDatetimeAndNumeric`to allow casting between Datetime and Numeric. The default value of the configuration is `false`.
Also, casting double/float type to timestamp should raise exceptions if there is overflow or the input is Nan/infinite.

This is for better adoption of ANSI SQL mode:
- As we did some data science, we found that many Spark SQL users are actually using `Cast(Timestamp as Numeric)` and `Cast(Numeric as Timestamp)`. There are also some usages of `Cast(Date as Numeric)`.
- The Spark SQL connector for Tableau is using this feature for DateTime math. e.g.
 `CAST(FROM_UNIXTIME(CAST(CAST(%1 AS BIGINT) + (%2 * 86400) AS BIGINT)) AS TIMESTAMP)`

So, having a new configuration can provide users with an alternative choice on turning on ANSI mode.
Bug: Allow merging DecimalTypes with different precision values 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Nov/21 23:01
Affects Version/s: 3.0.3

When merging DecimalTypes with different precision but the same scale, one would get the following error:
{code:java}
Failed to merge fields 'col' and 'col'. Failed to merge decimal types with incompatible precision 17 and 12	at org.apache.spark.sql.types.StructType$.$anonfun$merge$2(StructType.scala:652)
	at scala.Option.map(Option.scala:230)
	at org.apache.spark.sql.types.StructType$.$anonfun$merge$1(StructType.scala:644)
	at org.apache.spark.sql.types.StructType$.$anonfun$merge$1$adapted(StructType.scala:641)
	at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
	at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
	at org.apache.spark.sql.types.StructType$.merge(StructType.scala:641)
	at org.apache.spark.sql.types.StructType.merge(StructType.scala:550) {code}
 

We could allow merging DecimalType values with different precision if the scale is the same for both types since there should not be any data correctness issues as one of the types will be extended, for example, DECIMAL(12, 2) -> DECIMAL(17, 2); however, this is not the case for upcasting when the scale is different - this would depend on the actual values.

 

Repro code:
{code:java}
import org.apache.spark.sql.types._

val schema1 = StructType(StructField("col", DecimalType(17, 2)) :: Nil)
val schema2 = StructType(StructField("col", DecimalType(12, 2)) :: Nil)
schema1.merge(schema2) {code}
 

This also affects Parquet schema merge which is where this issue was discovered originally:
{code:java}
import java.math.BigDecimal
import org.apache.spark.sql.Row
import org.apache.spark.sql.types._

val data1 = sc.parallelize(Row(new BigDecimal("1234567890000.11")) :: Nil, 1)
val schema1 = StructType(StructField("col", DecimalType(17, 2)) :: Nil)

val data2 = sc.parallelize(Row(new BigDecimal("123456789.11")) :: Nil, 1)
val schema2 = StructType(StructField("col", DecimalType(12, 2)) :: Nil)

spark.createDataFrame(data2, schema2).write.parquet("/tmp/decimal-test.parquet")
spark.createDataFrame(data1, schema1).write.mode("append").parquet("/tmp/decimal-test.parquet")

// Reading the DataFrame fails
spark.read.option("mergeSchema", "true").parquet("/tmp/decimal-test.parquet").show()

>>>
Failed merging schema:
root
 |-- col: decimal(17,2) (nullable = true)

Caused by: Failed to merge fields 'col' and 'col'. Failed to merge decimal types with incompatible precision 12 and 17



{code}
Bug: Support LONG argument to the Spark SQL LIMIT clause
Status: Open | Priority: Major | Resolution: 
Created: 01/Nov/21 12:18
Affects Version/s: 3.2.0

Big data sets may exceed INT max thus Limit clause in Spark SQL `LIMIT <integer_expression>` should be expanded to support <long_expressions>. 

Thus
 `SELECT * FROM <table> LIMIT 20000000000` would run and not return an error.

 

See docs: [https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-limit.html]
Bug: pyspark.pandas.to_numeric() should support the errors option
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 01/Nov/21 18:51
Affects Version/s: 3.2.0

In regular pandas you can say to_numeric(errors='coerce'). But the errors option is not recognized by pyspark.pandas.

FYI, the errors option is recognized by pyspark.pandas.to_datetime()
Bug: Disable unsupported `ExtendedLevelDBTest` on `MacOS/aarch64`
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 22/Nov/21 03:18
Affects Version/s: 4.0.0

After SPARK-37272  and SPARK-37282,  we can manually add
{code:java}
-Dtest.exclude.tags=org.apache.spark.tags.ExtendedLevelDBTest,org.apache.spark.tags.ExtendedRocksDBTest {code}
when run mvn test or sbt test to disable unsupported UTs on Macos using Apple Silicon.

 

We can add a profile to and  activate this property automatically when run UTs on Macos using Apple Silicon.
Bug: Migrate _java2py to SparkSession
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 15/Nov/21 17:18
Affects Version/s: 3.2.0

{{_java2py()}} uses a deprecated method to create a SparkSession.
 
https://github.com/apache/spark/blob/2fe9af8b2b91d0a46782dd6fff57eca8609be105/python/pyspark/ml/common.py#L99
Bug: PySpark pmod function
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Nov/21 16:14
Affects Version/s: 3.2.0

Because Spark is built on the JVM, in PySpark, F.lit(-1) % F.lit(2) returns -1. However, the modulus is often desired instead of the remainder.

 

There is a [PMOD() function in Spark SQL|https://spark.apache.org/docs/latest/api/sql/#pmod], but [not in PySpark|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql.html#functions]. So at the moment, the two options for getting the modulus is to use F.expr("pmod(A, B)"), or create a helper function such as:
 
{code:java}
def pmod(dividend, divisor):
    return F.when(dividend < 0, (dividend % divisor) + divisor).otherwise(dividend % divisor){code}
 
 
Neither are optimal - pmod should be native to PySpark as it is in Spark SQL.
Bug: Add available_now to PySpark trigger function
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 12/Nov/21 20:23
Affects Version/s: 3.2.0

With the release of the new streaming query trigger called `AvailableNow` in v3.2.0, it would be helpful to also include the trigger as part of the pyspark API. Making the trigger a part of the pyspark API would make it clear that this trigger is an option for python developers. This change to the pyspark API should be accompanied by an update to the docs (similar to https://github.com/apache/spark/pull/34333)

 

Related Jira ticket - https://issues.apache.org/jira/browse/SPARK-36533

Related github PR - https://github.com/apache/spark/pull/33763
Bug: Do not split input file for Parquet reader with aggregate push down
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 06/Nov/21 01:44
Affects Version/s: 3.3.0

As a followup of [https://github.com/apache/spark/pull/34298/files#r734795801,] Similar to ORC aggregate push down, we can disallow split input files for Parquet reader as well. See original comment for motivation.
Bug: Avoid Block Manager registrations when Executor is shutting down
Status: In Progress | Priority: Minor | Resolution: 
Created: 17/Nov/21 08:52
Affects Version/s: 3.2.0

*Note:* Similar to SPARK-34949 and SPARK-35011, BlockManager will reregister itself while executor is shutting down.

*Problem:*

As describe in SPARK-35011, HeartbeatReceiver.expireDeadHosts() will not clean those BlockManager if the executor is killed with reason Executor heartbeat timed out. Executors could heartbeat timed out because of network issue, or some other reason like SPARK-20977

*Logs:*

{code:java}
// Driver Logs

21/11/13 05:06:20,999 WARN [dispatcher-event-loop-36] spark.HeartbeatReceiver:69 : Removing executor 3056 with no recent heartbeats: 350149 ms exceeds timeout 300000 ms
21/11/13 05:06:20,999 INFO [kill-executor-thread] cluster.YarnClientSchedulerBackend:57 : Requesting to kill executor(s) 3056
21/11/13 05:06:21,000 INFO [kill-executor-thread] cluster.YarnClientSchedulerBackend:57 : Actual list of executor(s) to be killed is 3056
21/11/13 05:06:21,000 INFO [dispatcher-event-loop-8] yarn.ApplicationMaster$AMEndpoint:57 : Driver requested to kill executor(s) 3056.
21/11/13 05:06:21,000 INFO [dispatcher-CoarseGrainedScheduler] cluster.YarnSchedulerBackend$YarnDriverEndpoint:57 : Asked to remove executor 3056 with reason Executor heartbeat timed out after 350149 ms
21/11/13 05:06:21,000 ERROR [dispatcher-CoarseGrainedScheduler] cluster.YarnScheduler:73 : Lost executor 3056 on executor_host: Executor heartbeat timed out after 350149 ms
21/11/13 05:06:21,041 WARN [dispatcher-CoarseGrainedScheduler] scheduler.TaskSetManager:69 : Lost task 262191.0 in stage 452627.0 (TID 245764597, executor_host, executor 3056): ExecutorLostFailure (executor 3056 exited caused by one of the running tasks) Reason: Executor heartbeat timed out after 350149 ms
21/11/13 05:06:21,067 WARN [dispatcher-CoarseGrainedScheduler] scheduler.TaskSetManager:69 : Lost task 259149.0 in stage 452627.0 (TID 245761130, executor_host, executor 3056): ExecutorLostFailure (executor 3056 exited caused by one of the running tasks) Reason: Executor heartbeat timed out after 350149 ms
21/11/13 05:06:22,068 INFO [dispatcher-BlockManagerMaster] storage.BlockManagerMasterEndpoint:57 : Trying to remove executor 3056 from BlockManagerMaster.
21/11/13 05:06:22,072 INFO [dispatcher-BlockManagerMaster] storage.BlockManagerMasterEndpoint:57 : Removing block manager BlockManagerId(3056, executor_host, 30504, None)
21/11/13 05:06:22,073 INFO [dag-scheduler-event-loop] storage.BlockManagerMaster:57 : Removed 3056 successfully in removeExecutor
21/11/13 05:06:22,962 INFO [dispatcher-BlockManagerMaster] storage.BlockManagerMasterEndpoint:57 : Registering block manager executor_host:30504 with 88.5 GiB RAM, BlockManagerId(3056, executor_host, 30504, None)


// Executor Logs

21/11/13 05:06:21,004 INFO [dispatcher-Executor] executor.YarnCoarseGrainedExecutorBackend:57 : Driver commanded a shutdown
21/11/13 05:06:22,215 INFO [block-manager-future-0] storage.BlockManagerMaster:57 : Registering BlockManager BlockManagerId(3056, executor_host, 30504, None)
{code}
Bug: Child stage using merged output or not should be based on the availability of merged output from parent stage
Status: Open | Priority: Minor | Resolution: 
Created: 13/Nov/21 02:48
Affects Version/s: 3.2.1

As discussed in the [thread|https://github.com/apache/spark/pull/34461#pullrequestreview-799701494] in SPARK-37023, during a stage retry, if parent stage has already generated merged output in the previous attempt, with current behavior, the child stage would not able to fetch the merged output, as this is controlled by dependency.shuffleMergeEnabled (see current implementation [here|https://github.com/apache/spark/blob/31b6f614d3173c8a5852243bf7d0b6200788432d/core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala#L134-L136]) during the stage retry.

Instead of using a single variable to control behavior at both mapper side (push side) and reducer side (using merged output), whether child stage uses merged output or not must only be based on whether merged output is available for it to use(as discussed [here|https://github.com/apache/spark/pull/34461#issuecomment-964557253]).
Bug: Add Scale Parameter to Floor and Ceil functions
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 27/Nov/21 20:01
Affects Version/s: 3.2.0

This feature is proposed in the PR : https://github.com/apache/spark/pull/34593

Currently we support Decimal RoundingModes : HALF_UP (round) and HALF_EVEN (bround). But we have use cases that needs RoundingMode.UP and RoundingMode.DOWN.

[https://stackoverflow.com/questions/34888419/round-down-double-in-spark/40476117]

[https://stackoverflow.com/questions/54683066/is-there-a-rounddown-function-in-sql-as-there-is-in-excel]

[https://stackoverflow.com/questions/48279641/oracle-sql-round-half]

 

Floor and Ceil functions helps to do this but it doesn't support the position of the rounding. Adding scale parameter to the functions would help us control the rounding positions. 

 

Snowflake supports `scale` parameter to `floor`/`ceil` :
{code:java}
FLOOR( <input_expr> [, <scale_expr> ] ){code}
REF:

[https://docs.snowflake.com/en/sql-reference/functions/floor.html]
Bug: Removing redundant label addition
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 18/Nov/21 12:33
Affects Version/s: 3.3.0

[https://github.com/apache/spark/blob/3d0663c45f900332ff1ea826f45cbf2c79eba3cd/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala#L145]

 

[https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala#L279-L283]

 

These labels are already covered in conf.labels.
Bug: Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warnings
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 09/Nov/21 12:37
Affects Version/s: 3.3.0

There are some compilation warning log as follows:
{code:java}
[WARNING] [Warn] /spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala:268: [deprecation @ org.apache.spark.sql.catalyst.util.RebaseDateTime.loadRebaseRecords.mapper.$anon | origin=com.fasterxml.jackson.module.scala.ScalaObjectMapper | version=2.12.1] trait ScalaObjectMapper in package scala is deprecated (since 2.12.1): ScalaObjectMapper is deprecated because Manifests are not supported in Scala3 {code}
We can refer to the recommendations of `jackson-module-scala` for fix
{code:java}
ScalaObjectMapper is deprecated because Manifests are not supported in Scala3, you might want to use ClassTagExtensions as a replacement {code}