dynamobi/dynamobi — Issue #860: [FRG-10] Mismatch in Java and Fennel Calc when casting -1.5 to integer
Author: dynamobi-build
Created: 2011-12-30 03:53:24+00:00
State: closed
Labels: FRG, Java Calc Codegen

[reporter="angel", created="Mon, 26 Dec 2005 12:17:50 -0500 (GMT-05:00)"]
Using fennel calc: cast(-1.5e0 as integer) gives -2

Using java calc: cast(-1.5e0 as integer) gives -1

Note: I'm doing cast(-1.5e0 as integer) to make sure that -1.5

is intepreted as a double instead of decimal.

See sample output using fennel and java calcs below:

0: jdbc:farrago:> alter system set "calcVirtualMachine" = 'CALCVM_FENNEL';

No rows affected (0.008 seconds)

0: jdbc:farrago:> values cast (1.75e0 as integer);

+---------+

| EXPR$0 |

+---------+

| 2 |

+---------+

1 row selected (0.324 seconds)

0: jdbc:farrago:> values cast (1.5e0 as integer);

+---------+

| EXPR$0 |

+---------+

| 2 |

+---------+

1 row selected (0.227 seconds)

0: jdbc:farrago:> values cast (-1.5e0 as integer);

+---------+

| EXPR$0 |

+---------+

| -2 |

+---------+

1 row selected (0.232 seconds)

0: jdbc:farrago:> values cast (-1.75e0 as integer);

+---------+

| EXPR$0 |

+---------+

| -2 |

+---------+

1 row selected (0.242 seconds)

0: jdbc:farrago:> alter system set "calcVirtualMachine" = 'CALCVM_JAVA';

No rows affected (0.043 seconds)

0: jdbc:farrago:> values cast (1.75e0 as integer);

+---------+

| EXPR$0 |

+---------+

| 2 |

+---------+

1 row selected (0.075 seconds)

0: jdbc:farrago:> values cast (1.5e0 as integer);

+---------+

| EXPR$0 |

+---------+

| 2 |

+---------+

1 row selected (0.08 seconds)

0: jdbc:farrago:> values cast (-1.5e0 as integer);

+---------+

| EXPR$0 |

+---------+

| -1 |

+---------+

1 row selected (0.083 seconds)

0: jdbc:farrago:> values cast (-1.75e0 as integer);

+---------+

| EXPR$0 |

+---------+

| -2 |

+---------+

1 row selected (0.067 seconds)