av9ash/GitBugs huggingface dataset:
Bug: Add support for classloader/resource isolation
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/Jun/23 12:50
Affects Version/s: 3.5.0

A current limitation of Scala UDFs is that a Spark cluster would only be able to support a single REPL at a time due to the fact that classloaders of different Spark Sessions (and therefore, Spark Connect sessions) aren't isolated from each other. Without isolation, REPL-generated classfiles as well as user-added JARs may conflict if there are multiple users of the cluster.

Thus, we need a mechanism to support isolated sessions (i.e isolated resources/classloader) so that each REPL user does not conflict with other users on the same cluster.
Bug: Add optional pattern for Catalog.listFunctions
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/Jun/23 09:48
Affects Version/s: 3.5.0

Currently, the syntax

{code:java}
SHOW FUNCTIONS LIKE pattern
{code}

supports a optional pattern.
But the Catalog.listFunctions missing the function.
In fact, the optional pattern is very useful.
Bug: Add optional pattern for Catalog.listTables
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 05/Jun/23 04:27
Affects Version/s: 3.5.0

Currently, the syntax

{code:java}
SHOW TABLES LIKE pattern
{code}

supports a optional pattern.
But the Catalog.listTables missing the function.
In fact, the optional pattern is very useful.
Bug: CacheManager refreshes the fileIndex unnecessarily
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Jun/23 18:30
Affects Version/s: 3.4.1

The CacheManager on this line [https://github.com/apache/spark/blob/680ca2e56f2c8fc759743ad6755f6e3b1a19c629/sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala#L372] uses a prefix based matching to decide which file index needs to be refreshed. However, that can be incorrect if the users have paths which are not subdirectories but share prefixes. For example, in the function below:

 
{code:java}
  private def refreshFileIndexIfNecessary(
      fileIndex: FileIndex,
      fs: FileSystem,
      qualifiedPath: Path): Boolean = {
    val prefixToInvalidate = qualifiedPath.toString
    val needToRefresh = fileIndex.rootPaths
      .map(_.makeQualified(fs.getUri, fs.getWorkingDirectory).toString)
      .exists(_.startsWith(prefixToInvalidate))
    if (needToRefresh) fileIndex.refresh()
    needToRefresh
  } {code}
{{If the prefixToInvalidate is s3://bucket/mypath/table_dir and the file index has one of the root paths as s3://bucket/mypath/table_dir_2/part=1, then the needToRefresh will be true and the file index gets refreshed unnecessarily. This is not just wasted CPU cycles but can cause query failures as well, if there are access restrictions to the path being refreshed.}}
Bug: Inconsistent path qualifying between catalog and data operations
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Jun/23 03:49
Affects Version/s: 3.2.4

For example
 * CREATE TABLE statement with relative LOCATION will infer schema from files from the directory relative to the current working directory and store the directory relative to the warehouse path. 
 * CTAS statement with relative LOCATION cannot assert empty root path as it checks the wrong path it will finally use.
 * DataframeWriter does not qualify the path before checking
Bug: Add JobTag APIs to SparkR SparkContext
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Jun/23 16:04
Affects Version/s: 3.5.0

Add APIs added in https://issues.apache.org/jira/browse/SPARK-43952 to SparkR:
 * {{SparkContext.addJobTag(tag: String): Unit}}
 * {{SparkContext.removeJobTag(tag: String): Unit}}
 * {{SparkContext.getJobTags(): Set[String]}}
 * {{SparkContext.clearJobTags(): Unit}}
 * {{SparkContext.cancelJobsWithTag(tag: String): Unit}}
Bug: Follow-ups for Jar/Classfile Isolation
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Jun/23 14:21
Affects Version/s: 3.5.0

Related to https://issues.apache.org/jira/browse/SPARK-44146 ([PR|https://github.com/apache/spark/pull/41701]), this ticket is for the general follow-ups mentioned by [~hvanhovell] [here.|https://github.com/apache/spark/pull/41701#issuecomment-1608577372]
Bug: Code-gen for build side outer shuffled hash join
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Jun/23 21:59
Affects Version/s: 3.5.0

Here, build side outer join means LEFT OUTER join with build left, or RIGHT OUTER join with build right.

As a followup for https://github.com/apache/spark/pull/41398/ SPARK-36612 (non-codegen build-side outer shuffled hash join), this task is to add code-gen for it.
Bug: Extract shared code from StructType
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 23/Jun/23 20:51
Affects Version/s: 3.5.0

The StructType has some methods that require CatalystParser and Catalyst expression. We are not planning to move the parser and expression to the shared module thus needs to do code split to share as much as code as possible between Scala client and Catalyst.
Bug: Add named argument support in parser for function call
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Jun/23 21:46
Affects Version/s: 3.5.0

Today, we are implementing named argument support for user defined functions, some built-in functions, and table-valued functions. For the first step towards building such a feature, we need to make some requisite changes in the parser. 

To accomplish this, in this issue, we plan to add some new syntax tokens to the parser in Spark. Changes will also be made in the abstract syntax tree builder as well to reflect these new tokens. Such changes will first be restricted to normal function calls (table value functions will be treated separately).
Bug: Add array_agg, array_size, cardinality, count_min_sketch,mask,named_struct,json_* to Scala and Python
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Jun/23 01:25
Affects Version/s: 3.5.0

Add array_agg, array_size, cardinality, count_min_sketch

Add following functions:

* array_agg
* array_size
* cardinality
* count_min_sketch
* named_struct
* json_array_length
* json_object_keys
* mask


  to:

* Scala API
* Python API
* Spark Connect Scala Client
* Spark Connect Python Client
Bug: Kafka Source v2 should return preferred locations
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Jun/23 18:21
Affects Version/s: 3.5.0

DSv2 Kafka streaming source seems to miss setting the preferred location, which may destroy the purpose of cache for Kafka consumer (connection) & fetched data.

For DSv1, we have set the preferred location in RDD.

For DSv2, we should provide the info. in input partition, but we don't add the information into KafkaBatchInputPartition.
Bug: Potential memory leak when temp views created from DF created by structured streaming
Status: Open | Priority: Major | Resolution: 
Created: 29/Jun/23 20:22
Affects Version/s: 2.4.8

If the user registers a temporary view from a dataframe created by Structured Streaming and tries to drop the temporary view via his original SparkSession then memory will be leaking.

The reason is Structured streaming has its own SparkSession (as a clone of the original SparkSession, for details see https://issues.apache.org/jira/browse/SPARK-26586 and [https://github.com/apache/spark/blob/branch-3.4/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala#L193-L194]) the created temporary view belongs the cloned SparkSession and the dropping of the temporary view must be done via the cloned SparkSession.

Example for the {*}memory leak{*}:
{noformat}
streamingDF.writeStream.foreachBatch { (batchDF: DataFrame, batchId: Long) =>
  val view = s“tempView_$batchId” 
  batchDF.createOrReplaceTempView(view)
  ...
  spark.catalog.dropTempView(view)
}
{noformat}
*Workaround* (the _dropTempView_ must be called on SparkSession accessed from dataframe created by streaming):
{noformat}
streamingDF.writeStream.foreachBatch { (batchDF: DataFrame, batchId: Long) =>
  val view = s“tempView_$batchId” 
  batchDF.createOrReplaceTempView(view)
  ...
  batchDF.sparkSession.catalog.dropTempView(view)
 }
{noformat}
h4. Example heap dump

The SparkSession with the leak:

!1.png|width=807,height=120!

The two SparkSession instances where the first one was is the original SparkSession created by the user and the second is the clone:
!2.png|width=813,height=157!
Bug: Exception when reading parquet file with TIME fields
Status: Open | Priority: Major | Resolution: 
Created: 24/Jun/23 11:41
Affects Version/s: 3.4.0

When one reads a parquet file containing TIME fields (either with INT32 or INT64 storage) and exception is thrown. From spark shell

 

{{> val df = spark.read.parquet("timeonly.parquet")}}

{color:#de350b}23/06/24 13:24:54 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)/ 1]{color}
{color:#de350b}org.apache.spark.sql.AnalysisException: Illegal Parquet type: INT32 (TIME(MILLIS,true)).{color}
{color:#de350b}    at org.apache.spark.sql.errors.QueryCompilationErrors$.illegalParquetTypeError(QueryCompilationErrors.scala:1762){color}
{color:#de350b}    at org.apache.spark.sql.execution.datasources.parquet.ParquetToSparkSchemaConverter.illegalType$1(ParquetSchemaConverter.scala:206){color}
{color:#de350b}    at org.apache.spark.sql.execution.datasources.parquet.ParquetToSparkSchemaConverter.$anonfun$convertPrimitiveField$2(ParquetSchemaConverter.scala:252){color}
{color:#de350b}    at scala.Option.getOrElse(Option.scala:189){color}
{color:#de350b}    at org.apache.spark.sql.execution.datasources.parquet.ParquetToSparkSchemaConverter.convertPrimitiveField(ParquetSchemaConverter.scala:224){color}
{color:#de350b}    at org.apache.spark.sql.execution.datasources.parquet.ParquetToSparkSchemaConverter.convertField(ParquetSchemaConverter.scala:187){color}
{color:#de350b}    at org.apache.spark.sql.execution.datasources.parquet.ParquetToSparkSchemaConverter.$anonfun$convertInternal$3(ParquetSchemaConverter.scala:147){color}
Bug: Isolate Spark Connect session/artifacts
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 22/Jun/23 17:15
Affects Version/s: 3.5.0

Following up on https://issues.apache.org/jira/browse/SPARK-44078, with the support for classloader isolation implemented, we can now utilise it to isolate Spark Connect sessions from each other. Here, isolation refers to isolation of artifacts from each Spark Connect session which enables us to have multi-user UDFs.
Bug: Add a parameter to determine the locality of local shuffle reader
Status: Open | Priority: Major | Resolution: 
Created: 29/Jun/23 08:34
Affects Version/s: 3.5.0

Local shuffle reader can achieve better performance with preferred locations. If we disable SHUFFLE_REDUCE_LOCALITY_ENABLE in queries that include reduce shuffles and local shuffles, local shuffle readers can not get preferred locations.

Add new parameter LOCAL_SHUFFLE_LOCALITY_ENABLE to determine whether to get the preferred locations of the current partitionSpec.
Bug: Where Clause throwing Resolved attribute(s) _metadata#398 missing from ... error
Status: Open | Priority: Major | Resolution: 
Created: 27/Jun/23 10:23
Affects Version/s: 3.3.1

i have 2 data frames called lt and rt, both with same schema and only 1 row, generated separately by our own curation logic, all the columns are either String, boolean or Timestamp, i am trying to compare them, and i am running a join on two like this 

var joinedDF = lt.join(rt, "Id")

after that, i am trying to compare them by schema fist and then by  each column, how many % of rows are same,

code is kindof like this

for (column <- lt.schema) {
     if (rt.columns.contains(column.name) &&
     column.dataType == rt.schema(column.name).dataType) {

      var matchCount = joinedCount
      if (column.dataType.typeName == "string") {
             matchCount = joinedDF.where((lt(column.name) <=> rt(column.name))).count}

else

.....

 

on the last line where i am running a where clause, it is throwing an error called AnalysisException Resolved attribute(s) _metadata#398 missing from ...., i don't even have this _metadata column anywhere in my dataframe at all

and i searched online people are saying it is a problem of join, i tried to change the colunm names in rt and joinedDF, both doesn't work, same error is still thrown, can anybody help here
Bug: Support an outer outer context in subquery resolution
Status: Open | Priority: Major | Resolution: 
Created: 28/Jun/23 22:09
Affects Version/s: 3.5.0

{code:python}
>>> sql("select * from range(8) t, lateral (select * from t) s")
Traceback (most recent call last):
...
pyspark.errors.exceptions.captured.AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] The table or view `t` cannot be found. Verify the spelling and correctness of the schema and catalog.
If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and catalog.
To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF EXISTS.; line 1 pos 49;
'Project [*]
+- 'LateralJoin lateral-subquery#0 [], Inner
   :  +- 'SubqueryAlias s
   :     +- 'Project [*]
   :        +- 'UnresolvedRelation [t], [], false
   +- SubqueryAlias t
      +- Range (0, 8, step=1, splits=None){code}
The subquery {{(select * from t)}} seems not looking the outer outer context and fails to resolve {{t}}.
Bug: Move resolveSelfJoinCondition to Analyzer
Status: Open | Priority: Major | Resolution: 
Created: 28/Jun/23 17:34
Affects Version/s: 3.5.0

Move the Joinwith object e.g. `ResolveSelfJoinCondition` into analyzer instead.
See more discussion from SPARK-43321 https://github.com/apache/spark/pull/40997/files#r1244509826
Bug: Table drop with Purge statement is not deleting the _temporary folder
Status: Open | Priority: Major | Resolution: 
Created: 28/Jun/23 13:30
Affects Version/s: 3.3.1

When saveAsTable is failed for some reason and `_temporary` folder created is not deleted with Drop table with purge statement. 

On high level , out data process look like below. 
 # Read data from external store using load 
 # Drop the table with purge just in case table exist already
 # Write Dataframe to hive using dataframe.write.saveMode(Overwrite).saveAsTable("my_table")

When Step 3 is failed for some reason and job is restarted because of yarm maxAttempts, Step is not deleting the _temporary folder created by Step 3 that is causing job to fail with below exception

org.apache.spark.sql.AnalysisException: Can not create the managed table("my_table"). The associated location('<hdfs path>') already exists.

This is never been a case in Spark2 because of "spark.sql.legacy.allowCreatingManagedTableUsingNonemptyLocation" config which is removed in Spark 3.
Bug: Dataset.selectExpr scope Session.active
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Jun/23 08:49
Affects Version/s: 3.4.0

{code:java}
//代码占位符
val clone = spark.cloneSession()
clone.conf.set("spark.sql.legacy.interval.enabled", "true")
clone.sql("select '2023-01-01'+ INTERVAL 1 YEAR as b").show()
clone.sql("select '2023-01-01' as a").selectExpr("a + INTERVAL 1 YEAR as b").show() {code}
The first one can be executed successfully, but the second one cannot be executed successfully.

Because selectExpr and sql use different sparkSession conf.
Bug: CTAS missing the child info on UI when groupSQLSubExecutionEnabled is enabled
Status: Open | Priority: Major | Resolution: 
Created: 27/Jun/23 16:36
Affects Version/s: 3.4.0

{code:sql}
create table tbl using parquet as select t1.id from range(10) as t1 join range(100) as t2 on t1.id = t2.id;
{code}
Enabled:
 !enabled.png! 
Disabled:
 !screenshot-1.png!
Bug: Add JobTag APIs to Java SparkContext
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Jun/23 01:08
Affects Version/s: 3.5.0

Add APIs added in https://issues.apache.org/jira/browse/SPARK-43952 to Java:

SparkContext.addJobTag(tag: String): Unit
SparkContext.removeJobTag(tag: String): Unit
SparkContext.getJobTags(): Set[String]
SparkContext.clearJobTags(): Unit
SparkContext.cancelJobsWithTag(tag: String): Unit
Bug: SparkR test failed with R 4.3.0
Status: Open | Priority: Major | Resolution: 
Created: 26/Jun/23 12:55
Affects Version/s: 3.5.0

run

 
{code:java}
build/sbt package -Psparkr
R/install-dev.sh; R/run-tests.sh {code}
 

 

 
{code:java}
══ Failed ══════════════════════════════════════════════════════════════════════
── 1. Error ('test_streaming.R:190:3'): PartitionBy ────────────────────────────
Error: Error in awaitTermination : streaming query error - Wrong basePath /var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T/RtmpDbPxOv/sparkr-testcf965edf703d.parquet for the root path: file:/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T/RtmpBFGJln/sparkr-testd75a4d1118fd.parquet/part-00000-bca8678d-70e2-42ef-abaf-5d5b56d46d62-c000.snappy.parquet
=== Streaming Query ===
Identifier: [id = 5a6a5714-ea05-473d-bdef-d30db92c8ef6, runId = bcfc36cd-bd4c-4b4a-9693-8ecafa70129a]
Current Committed Offsets: {}
Current Available Offsets: {FileStreamSource[file:/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T/RtmpDbPxOv/sparkr-testcf965edf703d.parquet]: {"logOffset":0}}


Current State: ACTIVE
Thread State: RUNNABLE


Logical Plan:
WriteToMicroBatchDataSourceV1 FileSink[/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T//RtmpDbPxOv/sparkr-testcf961f3f32b4.text], 5a6a5714-ea05-473d-bdef-d30db92c8ef6, [path=/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T//RtmpDbPxOv/sparkr-testcf961f3f32b4.text, checkpointLocation=append], Append
+- StreamingExecutionRelation FileStreamSource[file:/var/folders/2w/_w3fqbdn7pl8n4z2q6p6zcj80000gn/T/RtmpDbPxOv/sparkr-testcf965edf703d.parquet], [name#53978, age#53979, count#53980]


Backtrace:
    ▆
 1. ├─SparkR::awaitTermination(q, 5 * 1000) at test_streaming.R:190:2
 2. └─SparkR::awaitTermination(q, 5 * 1000)
 3.   └─SparkR:::handledCallJMethod(x@ssq, "awaitTermination", as.integer(timeout))
 4.     └─base::tryCatch(...)
 5.       └─base (local) tryCatchList(expr, classes, parentenv, handlers)
 6.         └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
 7.           └─value[[3L]](cond)
 8.             └─SparkR:::captureJVMException(e, method)


══ DONE ════════════════════════════════════════════════════════════════════════
Error: Test failures{code}
 

ENV:

 
{code:java}
R --version
R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin20 (64-bit)


R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.


java -version
openjdk version "1.8.0_372"
OpenJDK Runtime Environment (Zulu 8.70.0.23-CA-macos-aarch64) (build 1.8.0_372-b07)
OpenJDK 64-Bit Server VM (Zulu 8.70.0.23-CA-macos-aarch64) (build 25.372-b07, mixed mode) {code}
 

 

Run on M2 Max/MacOs 13.4.1
Bug: Add analyzer support of named arguments for built-in functions
Status: Open | Priority: Major | Resolution: 
Created: 26/Jun/23 16:53
Affects Version/s: 3.5.0

With the completion of the parser support for named arguments, we now have the foundations necessary for introducing changes in the analyzer that can resolve named argument expressions which are considered unevaluable. 

We wish to introduce function expression resolution for named arguments in the analyzer for built-in functions. This issue will track this change.
Bug: Support positional parameters in sql() by python connect client
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Jun/23 10:08
Affects Version/s: 3.5.0

As a follow-up to the parameterized query we added recently, we’d like to support positional parameters. This is part of the SQL standard and JDBC/ODBC protocol.

Example: update COFFEES set TOTAL = TOTAL + ? where COF_NAME = ?

Note that positional and named param marker cannot be used in the same query.
Bug: Add misc functions to Scala and Python
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Jun/23 00:53
Affects Version/s: 3.5.0

Add following functions:

* uuid
* aes_encrypt
* aes_decrypt
* sha
* input_file_block_length
* input_file_block_start
* reflect
* java_method
* version
* typeof
* stack
* random

to:

* Scala API
* Python API
* Spark Connect Scala Client
* Spark Connect Python Client
Bug: Maven test `ProductAggSuite` aborted
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 15/Jun/23 05:43
Affects Version/s: 3.5.0

run 

 
{code:java}
 ./build/mvn  -DskipTests -Pyarn -Pmesos -Pkubernetes -Pvolcano -Phive -Phive-thriftserver -Phadoop-cloud -Pspark-ganglia-lgpl  clean install
 build/mvn test -pl sql/catalyst     {code}
aborted

 
{code:java}
ProductAggSuite:
*** RUN ABORTED ***
  java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$
  at org.apache.spark.sql.catalyst.expressions.codegen.JavaCode$.variable(javaCode.scala:64)
  at org.apache.spark.sql.catalyst.expressions.codegen.JavaCode$.isNullVariable(javaCode.scala:77)
  at org.apache.spark.sql.catalyst.expressions.Expression.$anonfun$genCode$3(Expression.scala:200)
  at scala.Option.getOrElse(Option.scala:189)
  at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:196)
  at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.$anonfun$create$1(GenerateSafeProjection.scala:156)
  at scala.collection.immutable.List.map(List.scala:293)
  at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.create(GenerateSafeProjection.scala:153)
  at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.create(GenerateSafeProjection.scala:39)
  at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator.generate(CodeGenerator.scala:1369) {code}
Bug: Spark streaming: Add max pending microbatches conf to skip scheduling new mircobatch
Status: Closed | Priority: Major | Resolution: Won't Do
Created: 16/Jun/23 20:39
Affects Version/s: 3.4.0

In the case of uneven incoming rates and high scheduling delays, streaming will continue to add microbatches to the eventloop and submit the job to the job thread executor. Consequently, pending microbatches hold fewer offset ranges in Spark streaming Kafka if the kafka lag is less than the configured maximum per partition. 

We rely on the third-party service to add additional metadata to incoming records, and its response times remain constant regardless of microbatch size. So, Small microbatches can increase latencies further. An RDD's metadata is fetched during the transform phase in our case for various reasons, which is executed when micorbatch is scheduled. Our RDD transform on high level :
{code:java}
val dstreams = ...
dstreams.transform(rdd =>
 {   
  val uniqueItems = rdd.map(..).distinct.collect
  val metadata = getMedatada(uniqueItems)
  val rddWithMedatadata = rdd.map(...) // adds metadata  
   
  rddWithMedatadata
 })
{code}
 

Scheduling many small microbatches can be avoided by skipping new jobs when there are sufficient pending jobs in the queue. 

Proposed changes in _JobExecutor.scala_ on high level:
{code:java}
val maxPendingJobs = ssc.sc.conf.getInt("spark.streaming.maxPendingBatches", -1)

private def processEvent(event: JobGeneratorEvent): Unit = {
 logDebug("Got event " + event)
 event match {
  case GenerateJobs(time) =>
      if (maxPendingJobs == -1 || jobScheduler.getPendingTimes().size < maxPendingJobs){ 
            generateJobs(time)
       }else { 
         logWarning("Skipping JobGenerator at " + time)   // TODO: add pending times in queue to log.   
      }
  
  // other current cases
  case ...
   .....
 }
} {code}