apache/spark — pull request #2517 by scwf (2014-09-24 07:56:49+00:00)
Title: [SPARK-3676][Sql]spark sql hive test suite failed in JDK 1.6

https://issues.apache.org/jira/browse/SPARK-3676
spark sql hive test failed in jdk 1.6, you can replay this by set jdk version = 1.6.0_31
[info] - division *** FAILED ***
[info] Results do not match for division:
[info] SELECT 2 / 1, 1 / 2, 1 / 3, 1 / COUNT FROM src LIMIT 1
[info] == Parsed Logical Plan ==
[info] Limit 1
[info] Project (2 / 1) AS c_0#692,(1 / 2) AS c_1#693,(1 / 3) AS c_2#694,(1 / COUNT(1)) AS c_3#695
[info] UnresolvedRelation None, src, None
[info]
[info] == Analyzed Logical Plan ==
[info] Limit 1
[info] Aggregate [], [(CAST(2, DoubleType) / CAST(1, DoubleType)) AS c_0#692,(CAST(1, DoubleType) / CAST(2, DoubleType)) AS c_1#693,(CAST(1, DoubleType) / CAST(3, DoubleType)) AS c_2#694,(CAST(CAST(1, LongType), Doub
leType) / CAST(COUNT(1), DoubleType)) AS c_3#695]
[info] MetastoreRelation default, src, None
[info]
[info] == Optimized Logical Plan ==
[info] Limit 1
[info] Aggregate [], 2.0 AS c_0#692,0.5 AS c_1#693,0.3333333333333333 AS c_2#694,(1.0 / CAST(COUNT(1), DoubleType)) AS c_3#695
[info] Project []
[info] MetastoreRelation default, src, None
[info]
[info] == Physical Plan ==
[info] Limit 1
[info] Aggregate false, [], 2.0 AS c_0#692,0.5 AS c_1#693,0.3333333333333333 AS c_2#694,(1.0 / CAST(SUM(PartialCount#699L), DoubleType)) AS c_3#695
[info] Exchange SinglePartition
[info] Aggregate true, [], COUNT(1) AS PartialCount#699L
[info] HiveTableScan [], (MetastoreRelation default, src, None), None
[info]
[info] Code Generation: false
[info] == RDD ==
[info] c_0 c_1 c_2 c_3
[info] !== HIVE - 1 row(s) == == CATALYST - 1 row(s) ==
[info] !2.0 0.5 0.3333333333333333 0.002 2.0 0.5 0.3333333333333333 0.0020 (HiveComparisonTest.scala:370)
[info] - timestamp cast #1 *** FAILED ***
[info] Results do not match for timestamp cast #1:
[info] SELECT CAST(CAST(1 AS TIMESTAMP) AS DOUBLE) FROM src LIMIT 1
[info] == Parsed Logical Plan ==
[info] Limit 1
[info] Project CAST(CAST(1, TimestampType), DoubleType) AS c_0#995
[info] UnresolvedRelation None, src, None
[info]
[info] == Analyzed Logical Plan ==
[info] Limit 1
[info] Project CAST(CAST(1, TimestampType), DoubleType) AS c_0#995
[info] MetastoreRelation default, src, None
[info]
[info] == Optimized Logical Plan ==
[info] Limit 1
[info] Project 0.0010 AS c_0#995
[info] MetastoreRelation default, src, None
[info]
[info] == Physical Plan ==
[info] Limit 1
[info] Project 0.0010 AS c_0#995
[info] HiveTableScan [], (MetastoreRelation default, src, None), None
[info]
[info] Code Generation: false
[info] == RDD ==
[info] c_0
[info] !== HIVE - 1 row(s) == == CATALYST - 1 row(s) ==
[info] !0.001 0.0010 (HiveComparisonTest.scala:370)

this is because jdk has different logic to operate ```double```,
```System.out.println(1/500d)``` in different jdk get different result
jdk 1.6.0(_31) ---- 0.0020
jdk 1.7.0(_05) ---- 0.002
this lead to HiveQuerySuite failed when generate golden answer in jdk 1.7 and run tests in jdk 1.6, result did not matched