av9ash/GitBugs huggingface dataset:
Bug: StreamingQuery.processAllAvailable() blocks forever on queries containing mapGroupsWithState(GroupStateTimeout.ProcessingTimeTimeout())
Status: Open | Priority: Major | Resolution: 
Created: 15/Apr/22 18:38
Affects Version/s: 3.1.2

StreamingQuery.processAllAvailable() blocks forever when called on queries containing a mapGroupsWithState operation configured with GroupStateTimeout.ProcessingTimeTimeout().

 

I think processAllAvailable() should unblock when all incoming data has been processed AND when all existing groupStates do not have a current timeout specified.

 

Sample code to demonstrate this failure follows:
{code:java}
def demoSparkProcessAllAvailableBug() : Unit = {
    val localSpark = SparkSession
      .builder()
      .master("local[*]")
      .appName("demoSparkProcessAllAvailableBug")
      .config("spark.driver.host", "localhost")
      .getOrCreate()

    import localSpark.implicits._

    val demoDataStream = MemoryStream[BugDemo.NameNumberData](1, localSpark.sqlContext)
    demoDataStream.addData(BugDemo.NameNumberData("Alice", 1))
    demoDataStream.addData(BugDemo.NameNumberData("Bob", 2))
    demoDataStream.addData(BugDemo.NameNumberData("Alice", 3))
    demoDataStream.addData(BugDemo.NameNumberData("Bob", 4))

    // StreamingQuery.processAllAvailable() is successful when executing against NoTimeout,
    // but blocks forever when executing against EventTimeTimeout
    val timeoutTypes = List(GroupStateTimeout.NoTimeout(), GroupStateTimeout.ProcessingTimeTimeout())

    for (timeoutType <- timeoutTypes) {
      val totalByName = demoDataStream.toDF()
        .as[BugDemo.NameNumberData]
        .groupByKey(_.Name)
        .mapGroupsWithState(timeoutType)(BugDemo.summateRunningTotal)

      val totalByNameQuery = totalByName
        .writeStream
        .format("console")
        .outputMode("update")
        .start()

      println(s"${timeoutType} query starting to processAllAvailable()")
      totalByNameQuery.processAllAvailable()
      println(s"${timeoutType} query completed processAllAvailable()")

      totalByNameQuery.stop()
    }
  }
}

object BugDemo {
  def summateRunningTotal(name: String, input: Iterator[NameNumberData], groupState: GroupState[RunningTotal]): NameNumberData = {
    var currentTotal: Int = if (groupState.exists) {
      groupState.get.Total
    } else {
      0
    }

    for (nameNumberData <- input) {
      currentTotal += nameNumberData.Number
    }

    groupState.update(RunningTotal(currentTotal))

    NameNumberData(name, currentTotal)
  }

  case class NameNumberData(
    Name: String,
    Number: Integer
  )

  case class RunningTotal(
    Total: Integer
  )
} {code}
Bug: Windows, Spark 3.2.1: spark-shell command throwing this error: SparkContext: Error initializing SparkContext
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 06/Apr/22 14:29
Affects Version/s: 3.2.1

Can't start spark-shell on Windows for Spark 3.2.1. Downgrading Spark version to 3.1.3 fixed the problem. I need Pandas API on Spark, which is available on >3.2, so it's not a solution for me.

The bug and the workaround are described in the link below as well

https://stackoverflow.com/questions/69923603/spark-shell-command-throwing-this-error-sparkcontext-error-initializing-sparkc
Bug: optimizeSkewsInRebalancePartitions has performance issue
Status: Open | Priority: Major | Resolution: 
Created: 11/Apr/22 07:04
Affects Version/s: 3.4.0

How to reproduce this issue:
{code:sql}
CREATE TABLE t USING PARQUET
AS
SELECT
    /*+ REBALANCE */
	A.SESSION_START_DT
	, COALESCE(A.SITE_ID,0) AS SITE_ID
	, A.GUID
	, COALESCE(CAST(A.SESSION_SKEY AS BIGINT),0) AS SESSION_SKEY
	, COALESCE(CAST(A.SEQNUM AS INT),0) AS SEQNUM
	
	, COALESCE(A.IMP_PAGE_ID,0) AS IMP_PAGE_ID
	, COALESCE(A.PLACEMENT_ID,0) AS PLACEMENT_ID
	, A.PRODUCT_LINE_CODE
	, A.ALGORITHM_ID
	, A.MEID
	, A.ALGO_OUTPUT_ITEMS
	, A.CLICKS
	, A.GMV_7D
FROM big_partition_table A
WHERE
	DT BETWEEN DATE_FORMAT(DATE_SUB(CURRENT_DATE,11), 'yyyyMMdd') AND DATE_FORMAT(DATE_ADD(DATE_SUB(CURRENT_DATE,11),0), 'yyyyMMdd')
	AND TO_DATE(from_unixtime(unix_timestamp(A.SESSION_START_DT, 'yyyy/MM/dd'))) = DATE_SUB(CURRENT_DATE,11)
	AND ICFBOT = '00';
{code}

Enabling optimizeSkewsInRebalancePartitions takes more than 2 hours and the driver hangs:
 !enable.png! 
Disabling optimizeSkewsInRebalancePartitions takes only 29 minutes:
 !Disable.png!
Bug: Output parameter values of error classes in SQL style
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 05/Apr/22 20:01
Affects Version/s: 3.3.0

All parameters of error classes should be printed in SQL style. For example, string value "aaa" as 'aaa', a timestamp value as TIMESTAMP '2022-04-05 23:00:00' and so on.
Bug: Unify the AQE shuffle read canonicalized
Status: In Progress | Priority: Major | Resolution: 
Created: 14/Apr/22 06:55
Affects Version/s: 3.4.0

After canonicalized, the child of AQEShuffleReadExec will be a exchange instead of shuffle query stage. For better maintenance, we can simply override the isCanonicalizedPlan and let famework to check if the plan can be executed.
Bug: Regression in time executor takes to do work sometime after v3.0.1 ?
Status: Open | Priority: Major | Resolution: 
Created: 05/Apr/22 21:01
Affects Version/s: 3.2.1

Hello!

I'm sorry to trouble you with this, but I'm seeing a noticeable regression in performance when upgrading from 3.0.1 to 3.2.1 and I can't pin down why. I don't believe it is specific to my application since the upgrade to 3.0.1 to 3.2.1 is purely a configuration change. I'd guess it presents itself in my application due to the high volume of work my application does, but I could be mistaken.

The gist is that it seems like the executor actions I'm running suddenly appear to take a lot longer on Spark 3.2.1. I don't have any ability to test versions between 3.0.1 and 3.2.1 because my application was previously blocked from upgrading beyond Spark 3.0.1 by https://issues.apache.org/jira/browse/SPARK-37391 (which I helped to fix).

Any ideas what might cause this or metrics I might try to gather to pinpoint the problem? I've tried a bunch of the suggestions from [https://spark.apache.org/docs/latest/tuning.html] to see if any of those help, but none of the adjustments I've tried have been fruitful. I also tried to look in [https://spark.apache.org/docs/latest/sql-migration-guide.html] for ideas as to what might have changed to cause this behavior, but haven't seen anything that sticks out as being a possible source of the problem.

I have attached a graph that shows the drastic change in time taken by executor actions. In the image the blue and purple lines are different kinds of reads using the built-in JDBC data reader and the green line is writes using a custom-built data writer. The deploy to switch from 3.0.1 to 3.2.1 occurred at 9AM on the graph. The graph data comes from timing blocks that surround only the calls to dataframe actions, so there shouldn't be anything specific to my application that is suddenly inflating these numbers. The specific actions I'm invoking are: count() (but there's some transforming and caching going on, so it's really more than that); first(); and write().

The driver process does seem to be seeing more GC churn then with Spark 3.0.1, but I don't think that explains this behavior. The executors don't seem to have any problem with memory or GC and are not overutilized (our pipeline is very read and write heavy, less heavy on transformations, so executors tend to be idle while waiting for various network I/O).

 

Thanks in advance for any help!
Bug: Support switch inner join side for sort merge join
Status: In Progress | Priority: Major | Resolution: 
Created: 13/Apr/22 11:30
Affects Version/s: 3.4.0

For an inner join type SortMergeJoin, it always uses the left side as streamed side and right side as buffered side.

Accoirding to the implementaion of SortMergeJoin, we expect the buffered side to be:
 * smaller than streamed side
 * has less duplicate data

We do not know whether the join will be SortMergeJoin at logical phase, so it should do this selection at physcial phase.
Bug: DS V2 supports push down math functions
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 11/Apr/22 08:49
Affects Version/s: 3.4.0

Currently, Spark have a lot math functions of ANSI standard.
LN,
EXP,
POWER,
SQRT,
FLOOR,
CEIL,
WIDTH_BUCKET
DS V2 should supports push down these math functions
Bug: CLONE - Improve the test coverage for pyspark/*.py
Status: Resolved | Priority: Major | Resolution: Invalid
Created: 12/Apr/22 13:04
Affects Version/s: 3.3.0

Currently, there are several Python scripts under pyspark/ directory. (e.g. rdd.py, util.py, serializers.py, ...)

We could improve the test coverage by adding the missing tests for these scripts.
Bug: Improve the test coverage for PySpark/rddsampler.py
Status: Closed | Priority: Major | Resolution: Invalid
Created: 12/Apr/22 13:04
Affects Version/s: 3.3.0

Currently, PySpark test coverage is around 91% according to codecov report: [https://app.codecov.io/gh/apache/spark|https://app.codecov.io/gh/apache/spark]

Since there are still 9% missing tests, so I think it would be great to improve our test coverage.

Of course we might not target to 100%, but as much as possible, to the level that we can currently cover with CI.
Bug: Implement option to skip null values in symmetric hash impl of stream-stream joins
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Apr/22 18:38
Affects Version/s: 3.2.1

Implement option to skip null values in symmetric hash impl of stream-stream joins
 * In the symmetric has join state manager, we can receive entries with null values for a key and that caused the `removeByValue` and get iterators to fail and run into the NullPointerException.
 * This is possible if the state recovered is written from a old spark version or its corrupted on disk. Since we don't have a utility to query this state, we would like to provide a conf option to skip nulls for the symmetric hash impl in stream stream joins.
Bug: SparkContext init before SparkSession will cause hive table not found
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 10/Apr/22 08:36
Affects Version/s: 3.1.2

If we init SparkContext before SparkSession when using DataFrame to query hive table, if will throw table not found exception.
{code:java}
//代码占位符
val sparkContext = new SparkContext()

val sparkSession = SparkSession
  .builder
  .appName("SparkSession Test")
  .enableHiveSupport()
  .getOrCreate()

sparkSession.sql("select * from tableA"){code}
Bug: Fix translate metadata col filters
Status: In Progress | Priority: Major | Resolution: 
Created: 09/Apr/22 23:30
Affects Version/s: 3.3.0

Actually it can't be pushed down:
{noformat}
07:24:45.131 ERROR org.apache.spark.sql.execution.datasources.FileSourceStrategy: Pushed Filters: IsNotNull(_metadata.file_path),StringContains(_metadata.file_path,data/f0)
{noformat}
Bug: Formatting the log plan in AdaptiveSparkPlanExec
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Apr/22 07:52
Affects Version/s: 3.4.0


{code:java}
12:08:36.876 ERROR org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec: Plan changed from SortMergeJoin [key#13], [a#23], Inner
:- Sort [key#13 ASC NULLS FIRST], false, 0
:  +- ShuffleQueryStage 0
:     +- Exchange hashpartitioning(key#13, 5), ENSURE_REQUIREMENTS, [id=#110]
:        +- *(1) Filter (isnotnull(value#14) AND (value#14 = 1))
:           +- *(1) SerializeFromObject [knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData, true])).key AS key#13, staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData, true])).value, true, false, true) AS value#14]
:              +- Scan[obj#12]
+- Sort [a#23 ASC NULLS FIRST], false, 0
   +- ShuffleQueryStage 1
      +- Exchange hashpartitioning(a#23, 5), ENSURE_REQUIREMENTS, [id=#129]
         +- *(2) SerializeFromObject [knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData2, true])).a AS a#23, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData2, true])).b AS b#24]
            +- Scan[obj#22]
 to BroadcastHashJoin [key#13], [a#23], Inner, BuildLeft, false
:- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [id=#145]
:  +- ShuffleQueryStage 0
:     +- Exchange hashpartitioning(key#13, 5), ENSURE_REQUIREMENTS, [id=#110]
:        +- *(1) Filter (isnotnull(value#14) AND (value#14 = 1))
:           +- *(1) SerializeFromObject [knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData, true])).key AS key#13, staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData, true])).value, true, false, true) AS value#14]
:              +- Scan[obj#12]
+- ShuffleQueryStage 1
   +- Exchange hashpartitioning(a#23, 5), ENSURE_REQUIREMENTS, [id=#129]
      +- *(2) SerializeFromObject [knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData2, true])).a AS a#23, knownnotnull(assertnotnull(input[0, org.apache.spark.sql.test.SQLTestData$TestData2, true])).b AS b#24]
         +- Scan[obj#22]
{code}
Bug: Unexpected behaviour for "IN" operator in spark when null is involved in array
Status: Resolved | Priority: Major | Resolution: Not A Problem
Created: 05/Apr/22 16:49
Affects Version/s: 3.2.1

 

`IN`operator in spark.sql is giving unexpected results

1 in (null, 1) => true

1 in (null, 2) = null

I would have expected the second piece of code to throw false.

 

 
{code:java}
>>> spark.sql('SELECT 1 in (null, 1)').show()
+----------------+
|(1 IN (NULL, 1))|
+----------------+
|            true|
+----------------+

>>> spark.sql('SELECT 1 in (null, 2)').show()
+----------------+
|(1 IN (NULL, 2))|
+----------------+
|            null|
+----------------+
 {code}
Bug: Flaky test: ALSSuite.'ALS validate input dataset'
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 03/Apr/22 06:56
Affects Version/s: 3.3.0

- https://github.com/apache/spark/runs/5803714260?check_suite_focus=true

{code}
[info] ALSSuite:
...
[info] - ALS validate input dataset *** FAILED *** (2 seconds, 449 milliseconds)
[info]   Invalid Long: out of range "Job aborted due to stage failure: Task 0 in stage 100.0 failed 1 times, most recent failure: Lost task 0.0 in stage 100.0 (TID 348) (localhost executor driver): org.apache.spark.SparkArithmeticException: Casting 1231000000000 to int causes overflow. To return NULL instead, use 'try_cast'. If necessary set spark.sql.ansi.enabled to false to bypass this error.
{code}
Bug: spark thrift server issue: Length field is empty for varchar fields
Status: Open | Priority: Major | Resolution: 
Created: 01/Apr/22 17:39
Affects Version/s: 3.1.1

I am trying to read Data from Spark Thrift Server using SAS. In the table definition through DBeaver, I am seeing that *Length* field is empty only for fields with *VARCHAR* data type. I can see the length in the Data Type field as {*}varchar(32){*}. But that doesn't suffice my purpose as the SAS application taps into the Length field. Since, this field is not populated now, SAS is defaulting to the max size and as a result its becoming extremely slow. I get the length field populated in Hive.
Bug: Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin 
Status: In Progress | Priority: Major | Resolution: 
Created: 02/Apr/22 07:34
Affects Version/s: 3.4.0

Insert a bloom filter on one side of the join if it may be spill when sorting and the other side less than 100000000L rows.

A special case:

{code:java}
sql("set spark.sql.autoBroadcastJoinThreshold=10000")
sql("CREATE TABLE t1 using parquet AS SELECT id AS a, id AS b, id AS c FROM range(200000000L)")
sql("CREATE TABLE t2 using parquet AS SELECT id AS x, id AS y FROM range(8000000)")
sql("""SELECT a, b, c, x, y FROM t1 JOIN t2 ON t1.a = t2.x where t2.y = 1""").collect()
{code}
 !Special case.png!
Bug: Using EquivalentExpressions getEquivalentExprs function instead of getExprState at SubexpressionEliminationSuite
Status: In Progress | Priority: Major | Resolution: 
Created: 01/Apr/22 03:48
Affects Version/s: 3.1.0

Using EquivalentExpressions getEquivalentExprs function instead of getExprState at SubexpressionEliminationSuite, as spark in branch 3.1 has no getExprState fcuntion in EquivalentExpressions
Bug: Clean "classproperty" workaround in pyspark.sql.session once support for Python 3.8 is dropped
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 15/Apr/22 07:33
Affects Version/s: 3.2.1

*Context*

The proper bugfix of SPARK-38870 would be to use the @classmethod + @property decorators on the SparkSession.builder attribute to make it return a new builder every time like in Scala. 

However, @classmethod + @property decorators have only been made compatible since Python 3.9: [https://docs.python.org/3/howto/descriptor.html#class-methods]
> The code path for {{hasattr(type(self.f), '_{_}get{_}_')}} was added in Python 3.9 and makes it possible for [{{classmethod()}}|https://docs.python.org/3/library/functions.html#classmethod] to support chained decorators.

Moreover, @classmethod + @property waq also affected by a bug which has been been fixed in Python 3.9.6 (see also SPARK-38964) :
[https://github.com/python/cpython/pull/28838]

So we had to use a temporary workaround by defining a custom decorator named @classproperty to fix SPARK-38870. 

*What should be done*

Once support for Python 3.8 and "<=3.9.5" is officially dropped, we should remove this @classproperty workaround and replace it with @classmethod + @property (cf TODOs in the [pyspark.sql.session module|https://github.com/apache/spark/pull/36161/files]).

 

*Further details*

_Python 3.8 will end-of-life is 2024/10._
Bug: Update dataTables to 1.10.25 for security issue
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Apr/22 18:53
Affects Version/s: 3.3.0

https://nvd.nist.gov/vuln/detail/CVE-2020-28458 affects datatables up to 1.10.21 and we're on 1.10.20. It may or may not affect Spark, but updating to 1.10.25 at least should be easy
Bug: `DiskRowQueue` leak in `PythonForeachWriterSuite `
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 19/Apr/22 13:06
Affects Version/s: 3.4.0

`BufferTester`  defined in `PythonForeachWriterSuite` hold an `UnsafeRowBuffer` and 

`UnsafeRowBuffer.close()` was not called in this test. This will lead to fd leak
Bug: date_trunc and trunc both fail with format from column in inline table
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 21/Apr/22 22:27
Affects Version/s: 3.1.3

This fails:
{noformat}
create or replace temp view v1 as
select * from values ('week', timestamp'2012-01-01') as data(col1, col2);

select date_trunc(col1, col2) from v1;
{noformat}
It fails with a {{NullPointerException}}:
{noformat}
java.lang.NullPointerException: null
	at org.apache.spark.sql.catalyst.InternalRow$.$anonfun$getAccessor$8(InternalRow.scala:141) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.InternalRow$.$anonfun$getAccessor$8$adapted(InternalRow.scala:141) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.expressions.BoundReference.eval(BoundAttribute.scala:40) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.expressions.TruncInstant.evalHelper(datetimeExpressions.scala:2117) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.expressions.TruncInstant.evalHelper$(datetimeExpressions.scala:2112) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.expressions.TruncTimestamp.evalHelper(datetimeExpressions.scala:2277) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.expressions.TruncTimestamp.eval(datetimeExpressions.scala:2295) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.expressions.Alias.eval(namedExpressions.scala:157) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.apply(InterpretedMutableProjection.scala:97) ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
	at org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation$$anonfun$apply$47.$anonfun$applyOrElse$77(Optimizer.scala:1987) ~[spark-catalyst_2.12-3
{noformat}
 However, if you cache {{v1}}, then it works:
{noformat}
spark-sql> cache table v1;
Time taken: 2.086 seconds
spark-sql> select date_trunc(col1, col2) from v1;
2011-12-26 00:00:00
Time taken: 0.172 seconds, Fetched 1 row(s)
spark-sql> 
{noformat}
Both {{date_trunc}} and {{trunc}} exhibit this behavior.
Bug: Wrong comment in random matrix generator in spark-als algorithm 
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 07/Apr/22 09:53
Affects Version/s: 3.1.1

In algorithm Spark ALS we need initialize nonegative factor matricies for users and items. 

In ALS:

 
{code:java}
private def initialize[ID](
    inBlocks: RDD[(Int, InBlock[ID])],
    rank: Int,
    seed: Long): RDD[(Int, FactorBlock)] = {
  // Choose a unit vector uniformly at random from the unit sphere, but from the
  // "first quadrant" where all elements are nonnegative. This can be done by choosing
  // elements distributed as Normal(0,1) and taking the absolute value, and then normalizing.
  // This appears to create factorizations that have a slightly better reconstruction
  // (<1%) compared picking elements uniformly at random in [0,1].
  inBlocks.mapPartitions({ iter =>
    iter.map {
      case (srcBlockId, inBlock) =>
        val random: XORShiftRandom = new XORShiftRandom(byteswap64(seed ^ srcBlockId))
        val factors: Array[Array[Float]] = Array.fill(inBlock.srcIds.length) {
          val factor = Array.fill(rank)(random.nextGaussian().toFloat)
          val nrm: Float = blas.snrm2(rank, factor, 1)
          blas.sscal(rank, 1.0f / nrm, factor, 1)
          factor
        }
        (srcBlockId, factors)
    }
  }, preservesPartitioning = true)
} {code}
In the comments, the author writes that we are generating a matrix filled with positive numbers. In the code we use random.nextGaussian().toFloat. But if we look at the documentation of the nextGaussian method, we can see that it also returns negative numbers: 

 
{code:java}
/** 
* @return the next pseudorandom, Gaussian ("normally") distributed
 *         {@code double} value with mean {@code 0.0} and
 *         standard deviation {@code 1.0} from this random number
 *         generator's sequence
 */
synchronized public double nextGaussian() {
    // See Knuth, ACP, Section 3.4.1 Algorithm C.
    if (haveNextNextGaussian) {
        haveNextNextGaussian = false;
        return nextNextGaussian;
    } else {
        double v1, v2, s;
        do {
            v1 = 2 * nextDouble() - 1; // between -1 and 1
            v2 = 2 * nextDouble() - 1; // between -1 and 1
            s = v1 * v1 + v2 * v2;
        } while (s >= 1 || s == 0);
        double multiplier = StrictMath.sqrt(-2 * StrictMath.log(s)/s);
        nextNextGaussian = v2 * multiplier;
        haveNextNextGaussian = true;
        return v1 * multiplier;
    }
}
 {code}
 

The result is a matrix with negative values