av9ash/GitBugs huggingface dataset:
Bug: toPandas fails on simple query (collect() works)
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 19/Mar/20 05:02
Affects Version/s: 2.4.4

My pandas is 0.25.1.

I ran the following simple code (cross joins are enabled):

{code:python}
spark.sql('''
select t1.*, t2.* from (
  select explode(sequence(1, 3)) v
) t1 left join (
  select explode(sequence(1, 3)) v
) t2
''').toPandas()
{code}

and got a ValueError from pandas:

> ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Collect works fine:

{code:python}
spark.sql('''
select * from (
  select explode(sequence(1, 3)) v
) t1 left join (
  select explode(sequence(1, 3)) v
) t2
''').collect()
# [Row(v=1, v=1),
#  Row(v=1, v=2),
#  Row(v=1, v=3),
#  Row(v=2, v=1),
#  Row(v=2, v=2),
#  Row(v=2, v=3),
#  Row(v=3, v=1),
#  Row(v=3, v=2),
#  Row(v=3, v=3)]
{code}

I imagine it's related to the duplicate column names, but this doesn't fail:

{code:python}
spark.sql("select 1 v, 1 v").toPandas()
# v	v
# 0	1	1
{code}

Also no issue for multiple rows:

spark.sql("select 1 v, 1 v union all select 1 v, 2 v").toPandas()

It also works when not using a cross join but a janky programatically-generated union all query:

{code:python}
cond = []
for ii in range(3):
    for jj in range(3):
        cond.append(f'select {ii+1} v, {jj+1} v')
spark.sql(' union all '.join(cond)).toPandas()
{code}

As near as I can tell, the output is identical to the explode output, making this issue all the more peculiar, as I thought toPandas() is applied to the output of collect(), so if collect() gives the same output, how can toPandas() fail in one case and not the other? Further, the lazy DataFrame is the same: DataFrame[v: int, v: int] in both cases. I must be missing something.
Bug: Fix flaky test: WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 25/Mar/20 00:49
Affects Version/s: 3.1.0

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120284/testReport/junit/org.apache.spark.scheduler/WorkerDecommissionSuite/verify_a_task_with_all_workers_decommissioned_succeeds/

Error Message
java.util.concurrent.TimeoutException: Futures timed out after [2 seconds]
Stacktrace
sbt.ForkMain$ForkError: java.util.concurrent.TimeoutException: Futures timed out after [2 seconds]
	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:259)
	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:187)
	at org.apache.spark.SimpleFutureAction.result(FutureAction.scala:130)
	at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:295)
	at org.apache.spark.scheduler.WorkerDecommissionSuite.$anonfun$new$3(WorkerDecommissionSuite.scala:73)
	at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
	at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
	at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
	at org.scalatest.Transformer.apply(Transformer.scala:22)
	at org.scalatest.Transformer.apply(Transformer.scala:20)
	at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186)
	at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:151)
	at org.scalatest.FunSuiteLike.invokeWithFixture$1(FunSuiteLike.scala:184)
	at org.scalatest.FunSuiteLike.$anonfun$runTest$1(FunSuiteLike.scala:196)
	at org.scalatest.SuperEngine.runTestImpl(Engine.scala:286)
	at org.scalatest.FunSuiteLike.runTest(FunSuiteLike.scala:196)
	at org.scalatest.FunSuiteLike.runTest$(FunSuiteLike.scala:178)
	at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:58)
Bug: Failed to register signal handler for PWR
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 02/Mar/20 13:33
Affects Version/s: 3.1.0

I've just tried to test something on standalone mode but the application fails.

Environment:
 * MacOS Catalina 10.15.3 (19D76)
 * Scala 2.12.10
 * Java 1.8.0_241-b07

Steps to reproduce:
 * Compile Spark (mvn -DskipTests clean install -Dskip)
 * ./sbin/start-master.sh
 * ./sbin/start-slave.sh spark://host:7077
 * submit an empty application

Error:
{code:java}
20/03/02 14:25:44 INFO SignalUtils: Registering signal handler for PWR
20/03/02 14:25:44 WARN SignalUtils: Failed to register signal handler for PWR
java.lang.IllegalArgumentException: Unknown signal: PWR
	at sun.misc.Signal.<init>(Signal.java:143)
	at org.apache.spark.util.SignalUtils$.$anonfun$register$1(SignalUtils.scala:64)
	at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:86)
	at org.apache.spark.util.SignalUtils$.register(SignalUtils.scala:62)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend.onStart(CoarseGrainedExecutorBackend.scala:85)
	at org.apache.spark.rpc.netty.Inbox.$anonfun$process$1(Inbox.scala:120)
	at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:203)
	at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
	at org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:75)
	at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:41)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
{code}
Bug: Add unit tests TypeCoercion.findTypeForComplex and Cast.canCast about null <> complex types
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Mar/20 12:13
Affects Version/s: 3.0.0

This JIRA targets to describe to add the unittests. This is rather a followup work at SPARK-31166; however, this JIRA targets to include the tests with struct type and array types against null types.
Bug: Support Quick sample in RDD
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 13/Mar/20 03:03
Affects Version/s: 3.1.0

RDD.sample use the function of *filter* to pick up the data we need. It means that if the raw data is very huge, we must spend too much time reading it. We can filter the raw partition to speed up the processing of sample.


{code:java}
  override def compute(splitIn: Partition, context: TaskContext): Iterator[U] = {
    val split = splitIn.asInstanceOf[PartitionwiseSampledRDDPartition]
    val thisSampler = sampler.clone
    thisSampler.setSeed(split.seed)
    thisSampler.sample(firstParent[T].iterator(split.prev, context))
  }
{code}
Bug: unix_timestamp() function returning NULL values for corner cases (daylight saving)
Status: Resolved | Priority: Minor | Resolution: Cannot Reproduce
Created: 17/Mar/20 16:04
Affects Version/s: 2.3.4

Running below code does not return 4 values due to one timestamp not being correctly created (possible problem with daylight saving):
{code:java}
spark.sql("""SELECT from_unixtime(unix_timestamp('2020-03-08 01:00:00'), 'yyyyMMdd') t1, from_unixtime(unix_timestamp('2020-03-08 02:00:00'), 'yyyyMMdd') t2, from_unixtime(unix_timestamp('2020-03-08 03:00:00'), 'yyyyMMdd') t3, from_unixtime(unix_timestamp('2020-03-08 04:00:00'), 'yyyyMMdd') t4""").show

+--------+----+--------+--------+
|      t1|  t2|      t3|      t4|
+--------+----+--------+--------+
|20200308|null|20200308|20200308|
+--------+----+--------+--------+{code}
This unexpected NULL value caused us problems as we did not expect it.
Bug: RecordBinaryComparator Tests failing on Big Endian Platform (s390x)
Status: Resolved | Priority: Minor | Resolution: Incomplete
Created: 17/Mar/20 14:24
Affects Version/s: 2.4.5

Failing Test Cases in the RecordBinaryComparatorSuite:
 * testBinaryComparatorWhenSubtractionIsDivisibleByMaxIntValue
 * testBinaryComparatorWhenSubtractionCanOverflowLongValue

Test cases failed after the change related to:

[Github Pull Request #26548|https://github.com/apache/spark/pull/26548#issuecomment-554645859]

Test Case: testBinaryComparatorWhenSubtractionIsDivisibleByMaxIntValue
 * Fails due to changing the compare from `<` to `>` as the test condition (In little endian this is valid when the bytes are reversed, but not for big endian)

Test Case: testBinaryComparatorWhenSubtractionCanOverflowLongValue
 * Fails due to using Long.compareUnsigned
 * Possible Fix: Use signed compare for big endian platforms.
Bug: Remove pydocstyle tests
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 14/Mar/20 08:22
Affects Version/s: 3.0.0

pydocstyle tests have been running neither on Jenkins nor on Github.

We also seem to be in a [bad place|https://github.com/apache/spark/pull/27912#issuecomment-599167117] to re-enable them.
Bug: [DEPLOY] Pack the user jars when submitting Spark Application
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 03/Mar/20 07:30
Affects Version/s: 3.0.0

Nowadays, Spark only pack the jars under $SPARK_HOME/jars.
How about packing the user jars when submitting Spark application?
Sometimes, user may involve lots of jars expect spark libs.
I think it can reduce the pressure for HDFS and nodemanager(localizer).
Bug: Non-nullable null type should not coerce to nullable type
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Mar/20 11:13
Affects Version/s: 3.1.0

{code}
scala> spark.range(10).selectExpr("array()").printSchema()
root
 |-- array(): array (nullable = false)
 |    |-- element: null (containsNull = false)


scala> spark.range(10).selectExpr("concat(array()) as arr").printSchema()
root
 |-- arr: array (nullable = false)
 |    |-- element: null (containsNull = false)


scala> spark.range(10).selectExpr("concat(array(), array(1)) as arr").printSchema()
root
 |-- arr: array (nullable = false)
 |    |-- element: integer (containsNull = true)
{code}

The last case should not coerce to nullable type.
Bug: DataFrameStatFunctions API is not consistent with respect to Column type
Status: Resolved | Priority: Minor | Resolution: Incomplete
Created: 14/Mar/20 22:36
Affects Version/s: 2.4.4

Some functions from {{org.apache.spark.sql.DataFrameStatFunctions}} class accepts {{org.apache.spark.sql.Column}} as an argument:
 * {{bloomFilter}}
 * {{countMinSketch}}
 * {{sampleBy}}

When the rest of the functions accept only {{String}} (or collections of {{String}}'s respectively):
 * {{approxQuantile}}
 * {{corr}}
 * {{cov}}
 * {{crosstab}}
 * {{freqItems}}
Bug: Removing redundant rules in the output of Frequent Pattern Growth Algorithm
Status: Resolved | Priority: Minor | Resolution: Incomplete
Created: 09/Mar/20 13:00
Affects Version/s: 2.4.5

Will implement the is.redundant() function similar to the one here: [https://rdrr.io/cran/arules/man/is.redundant.html]

By definition:

A rule is redundant if a more general rules with the same or a higher confidence exists. That is, a more specific rule is redundant if it is only equally or even less predictive than a more general rule.

As FP Growth is an exhaustive algorithm, many of the rules it produces are redundant. Therefore there is merit in implementing this function to spark. This not only reduces the total number of rules produced in the output, but also produces better rules.
Bug: Add support for sparse matrix multiplication
Status: Resolved | Priority: Minor | Resolution: Incomplete
Created: 11/Mar/20 12:01
Affects Version/s: 2.4.5

MLlib does not currently support multiplication of sparse matrices. When multiplying block matrices with sparse blocks, the sparse blocks are first converted to dense matrices. This leads to large increases in memory utilization for certain problems.

I'd like to propose adding support for local sparse matrix multiplication to MLlib, as well as local dense-sparse matrix multiplication. With these changes, the case clause which converts sparse blocks to dense matrices in the block matrix multiply method could be removed.

One question is whether the result of sparse-sparse matrix multiplication should be sparse or dense, since the product of two sparse matrices can be quite dense depending on the matrices. I propose returning a sparse matrix, however, and letting the application convert the result to a dense matrix if necessary. There is some precedent for this with the block matrix add method, which returns sparse matrix blocks even when adding a sparse matrix block to a dense matrix block.

As for the implementation, one option would be to leverage Breeze's existing sparse matrix multiplication, as MLlib currently does for matrix addition. Another would be to add support for sparse-sparse multiplication to the BLAS wrapper, which would be consistent with the sparse-dense multiplication implementation and could support a more efficient routine for transposed matrices (as Breeze does not support transposed matrices). The exact algorithm would follow that laid out in ["Sparse Matrix Multiplication Package (SMMP)"|https://www.i2m.univ-amu.fr/perso/abdallah.bradji/multp_sparse.pdf].

This would likely not be a huge change but would take some time to test and benchmark properly, so before I put up a code diff I would be curious to know:
* Is there any interest in supporting this functionality in MLlib?
* Is there a preference for the return type of sparse-sparse multiplication? (i.e. sparse or dense)
* Is there a preference for the implementation? (Breeze vs a built-in one)

Some tickets which include related functionality or identified this particular issue but never solved it: SPARK-16820, SPARK-3418.
Bug: Refactor how we track Python test/build dependencies
Status: In Progress | Priority: Minor | Resolution: 
Created: 16/Mar/20 15:37
Affects Version/s: 3.1.0

Ideally, we should have a single place to track Python development dependencies and reuse it in all the relevant places: developer docs, Dockerfile, and GitHub CI. Where appropriate, we should pin dependencies to ensure a reproducible Python environment.
Bug: Support foldable schemas by `from_json`
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 03/Mar/20 16:26
Affects Version/s: 3.0.0

Currently, Spark accepts only literals or schema_of_json w/ literal input as the schema parameter of from_json. And it fails on any foldable expressions, for instance:
{code:sql}
spark-sql> select from_json('{"id":1, "city":"Moscow"}', replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', ''));
Error in query: Schema should be specified in DDL format as a string literal or output of the schema_of_json function instead of replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', '');; line 1 pos 7
{code}
There are no reasons to restrict users by literals. The ticket aims to support any foldable schemas by from_json().
Bug: Fast fail while fetching shuffle data unsuccessfully
Status: In Progress | Priority: Minor | Resolution: 
Created: 09/Mar/20 11:42
Affects Version/s: 3.1.0

When fetching shuffle data unsuccessfully, we put a FailureFetchResult into results(a linkedBlockingQueue) and wait it to be taken.
Then a FetchFailedException would be thrown.

In fact, we can fast fail the task once fetching shuffle data unsuccessfully.
Bug: Add -XX:MaxDirectMemorySize jvm options in yarn mode
Status: Resolved | Priority: Minor | Resolution: Invalid
Created: 26/Mar/20 06:22
Affects Version/s: 3.0.0

Current memory composition `amMemory` + `amMemoryOverhead`
{code:java}
val capability = Records.newRecord(classOf[Resource])
capability.setMemory(amMemory + amMemoryOverhead)
capability.setVirtualCores(amCores)
if (amResources.nonEmpty) {
 ResourceRequestHelper.setResourceRequests(amResources, capability)
}
logDebug(s"Created resource capability for AM request: $capability")
{code}
{code:java}
 // Add Xmx for AM memory 
javaOpts += "-Xmx" + amMemory + "m"
{code}
It is possible that the physical memory of the container exceeds the limit and is killed by yarn.
I suggest setting `-XX:MaxDirectMemorySize` here
{code:java}
// Add Xmx for AM memory
javaOpts += "-Xmx" + amMemory + "m"
javaOpts += s"-XX:MaxDirectMemorySize=${amMemoryOverhead}m"{code}
Bug: Separates different categories of applications
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Mar/20 14:01
Affects Version/s: 3.0.0

The current application defaults to the SPARK type. 
In fact, different types of applications have different characteristics and are suitable for different scenarios.For example: SPAKR-SQL, SPARK-STREAMING.
I recommend distinguishing them by the parameter `spark.yarn.applicationType` so that we can more easily manage and maintain different types of applications.
Bug: Flaky test: org.apache.spark.sql.hive.thriftserver.CliSuite
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 27/Mar/20 18:29
Affects Version/s: 3.1.0


{code:java}
Caused by: MetaException(message:Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=/home/jenkins/workspace/SparkPullRequestBuilder/target/tmp/spark-17bf6d71-1e68-4e56-b656-f1b2fd2e15fb;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
2020-03-27 09:20:11.949 - stderr> java.sql.SQLException: Failed to create database '/home/jenkins/workspace/SparkPullRequestBuilder/target/tmp/spark-17bf6d71-1e68-4e56-b656-f1b2fd2e15fb', see the next exception for details.
{code}



{code:java}
Caused by: ERROR XBM0A: The database directory '/home/jenkins/workspace/SparkPullRequestBuilder@4/target/tmp/spark-84c8ff0e-214f-416c-9d44-ab19f864a79b' exists. However, it does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this database. You may want to delete this directory and try creating the database again.
{code}
Bug: Implement PowerTransform
Status: In Progress | Priority: Minor | Resolution: 
Created: 18/Mar/20 07:15
Affects Version/s: 3.1.0

{color:#5a6e5a}Power transforms are a family of parametric, monotonic transformations
{color}{color:#5a6e5a}that are applied to make data more Gaussian-like. This is useful for
{color}{color:#5a6e5a}modeling issues related to heteroscedasticity (non-constant variance),
{color}{color:#5a6e5a}or other situations where normality is desired.{color}
Bug: PairRDD support aggregateByKeyWithinPartitions
Status: Resolved | Priority: Minor | Resolution: Not A Problem
Created: 18/Mar/20 09:13
Affects Version/s: 3.1.0

When implementing `RobustScaler`, I was looking for a way to guarantee that the {{QuantileSummaries in {{}}{{aggregateByKey}}{{}}}} are compressed before network communication.

Then I only found a tricky method to work around (however not applied), and there was no method for this.
Bug: Enhance RpcTimeoutException Log Message
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Mar/20 12:52
Affects Version/s: 3.1.0

Application log: 
{code:java}
Failed to process batch org.apache.spark.rpc.RpcTimeoutException: Futures timed out after [800 seconds]. This timeout is controlled by spark.network.timeout:
org.apache.spark.rpc.RpcTimeout.org
$apache$spark$rpc$RpcTimeout$$createRpcTimeoutException(RpcTimeout.scala:47)
{code}
Driver log:
{code:java}
[block-manager-ask-thread-pool-149] WARN org.apache.spark.storage.BlockManagerMaster - Failed to remove RDD 25344 - Cannot receive any reply from null in 800 seconds. This timeout is controlled by spark.network.timeout org.apache.spark.rpc.RpcTimeoutException: Cannot receive any reply from null in 800 seconds. This timeout is controlled by spark.network.timeout{code}
The log message does not provide RpcAddress of the destination RpcEndpoint. It is due to 
{noformat}
* The `rpcAddress` may be null, in which case the endpoint is registered via a client-only
* connection and can only be reached via the client that sent the endpoint reference.{noformat}
 Solution:

using rpcAdress from client of the NettyRpcEndpoingRef once such endpoint resides in client mode.
Bug: Do not upper case schedule mode
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 27/Mar/20 08:16
Affects Version/s: 3.1.0

In `TaskSchedulerImpl`, Spark will upper schedule mode  `SchedulingMode.withName(schedulingModeConf.toUpperCase(Locale.ROOT))`.
But at other place, Spark does not. Such as  [AllJobsPage|https://github.com/apache/spark/blob/5945d46c11a86fd85f9e65f24c2e88f368eee01f/core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala#L304].
We should have the same behavior.
Bug: Fix wrong examples and help messages for Kinesis integration
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 28/Mar/20 13:47
Affects Version/s: 2.3.4

There are some minor mistakes in the examples and the help messages for Kinesis integration. For example, {{KinesisWordCountASL.scala}} takes three arguments but its example is taking four, while {{kinesis_wordcount_asl.py}} takes four but its example is taking three.
Bug: Migrate `DateTimeTestUtils` from `TimeZone` to `ZoneId`
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 26/Mar/20 19:16
Affects Version/s: 3.0.0

Currently, Spark SQL's date-time expressions and functions are ported on Java 8 time API but tests still use old time APIs. In particular, DateTimeTestUtils exposes functions that accept only TimeZone instances. This is inconvenient, and CPU consuming because need to convert TimeZone instances to ZoneId instances via strings (zone ids). The tickets aims to replace TimeZone parameters of DateTimeTestUtils functions by ZoneId type.
Bug: Improve file listing for ViewFileSystem
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 05/Mar/20 01:58
Affects Version/s: 3.1.0

https://issues.apache.org/jira/browse/SPARK-27801 has improved file listing for DistributedFileSystem, where {{InMemoryFileIndex.listLeafFiles}} makes use of DistributedFileSystem's one single {{listLocatedStatus}} to namenode. This ticket intends to improve the case where ViewFileSystem is used to manage multiple DistributedFileSystems. It has also overridden the {{listLocatedStatus}} method by delegating to the filesystem it resolves to, e.g. DistributedFileSystem.
Bug: Refactor the on-click timeline action in streagming-page.js
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Mar/20 02:49
Affects Version/s: 3.1.0

streaming-page.js is used for Streaming page and Structured Steraming page but the implementation of on-click timeline action is strongly dependent on Streaming page.
So let's refactor to remove the dependency.
Bug: Server-side processing of History UI list of applications
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 19/Mar/20 20:58
Affects Version/s: 2.3.0

Loading the list of applications in the History UI does not scale well with a large number of applications. Fetching and rendering the list for 10k+ applications takes over a minute (much longer for more applications) and tends to freeze the browser.

Using `spark.history.ui.maxApplications` is not a great solution, because (as the name implies), it limits the number of applications shown in the UI, which hinders usability of the History Server.

A solution would be to use server [side processing of the DataTable|https://datatables.net/examples/data_sources/server_side]. This would limit amount of data sent to the client and processed by the browser.

This proposed change plays nicely with KVStore abstraction implemented in SPARK-18085, which was supposed to solve some of the scalability issues. It could also definitely solve History UI scalability issues reported for example in SPARK-21254, SPARK-17243, SPARK-17671
Bug: Introduce PushProjectThroughLimit
Status: Open | Priority: Minor | Resolution: 
Created: 19/Mar/20 14:36
Affects Version/s: 3.0.0

Currently the {{CollapseProject}} rule does many things: not only it collapses stacked projects, but also pushes down projects into limits, windows, etc. In this PR we factored out rules from {{CollapseProject}} that were pushing projects into limits and introduced a new rule called {{PushProjectThroughLimit.}}