Repository: apytypes/apytypes (language: python)

Issue:
Deviations from Berkeley TestFloat
Some deviations and bugs in floating-point arithmetic were found when testing against Berkeley TestFloat:
* [x] (#408) As already known, the sign of zero is incorrect as it doesn't consider the quantization mode. 
* [x] (#407) The result sometimes becomes infinity instead of the maximum normal number for certain quantization modes. This makes sense when reading the definitions again, e.g. for TO_POS (from the 754 standard): "the result shall be the format’s floating-point number (possibly +∞) closest to and no less than the infinitely precise result". The result should therefore never become negative infinity for TO_POS.
* [x] (#428) f16_mul sometimes fails for directed quantization modes, e.g. the following for TO_POS:
```Python
APyFloat(1, 1, 1023, 5, 10) * APyFloat(1, 26, 80, 5, 10)
# Becomes APyFloat(sign=0, exp=13, man=79, exp_bits=5, man_bits=10)
# and not APyFloat(sign=0, exp=13, man=80, exp_bits=5, man_bits=10)
```
* [x] (#411) f32_mul sometimes results in a subnormal number instead of zero, however this isn't seen for f64_mul or f16_mul):
```Python
APyFloat(0, 53, 32895, 8, 23) * APyFloat(1, 1, 1999049, 8, 23)
# Becomes APyFloat(sign=1, exp=0, man=10184, exp_bits=8, man_bits=23)
# and not APyFloat(sign=1, exp=0, man=0, exp_bits=8, man_bits=23)
```
* [x] Added bullet point. The mantissa is sometimes wrong when casting from a larger format to a smaller, similar to the previous bullet. See comment below.

These were found from running
```cmd
python run_berkeley_cases.py -op all -qm all -s 1 -l 1
```
but dedicated test cases should of course be made while fixing these.


Discussion / hints:
Issues in `cast` has been found from Berkeley TestFloat, where the mantissa sometimes becomes very wrong. This is perhaps similar to the bug that caused bullet point four, when a large shift caused undefined behaviour. 
```Python
APyFloat(sign=0, exp=103, man=0, exp_bits=8, man_bits=23).cast(5, 10)
# Becomes APyFloat(sign=0, exp=0, man=8192, exp_bits=5, man_bits=10)
# Instead of APyFloat(sign=0, exp=0, man=1, exp_bits=5, man_bits=10)
```
At the same time, $8192$ equals $2^{13}$ which may not be coincidence since $13$ is the difference in mantissa bit length.

Fix patch:
diff --git a/src/apyfloat.cc b/src/apyfloat.cc
index a179bcd83..59ff9fcf4 100644
--- a/src/apyfloat.cc
+++ b/src/apyfloat.cc
@@ -224,8 +224,12 @@ APyFloat APyFloat::_checked_cast(
         res.exp = 0;
         // Cast mantissa
         res.man = prev_man;
-        std::uint8_t man_bits_delta = 1 - new_exp;
-        res.cast_mantissa_subnormal(man_bits_delta, quantization);
+        const int man_bits_delta = 1 - new_exp + (man_bits - new_man_bits);
+        if (man_bits_delta > 0) {
+            res.cast_mantissa_subnormal(man_bits_delta, quantization);
+        } else {
+            res.man <<= -man_bits_delta;
+        }
         return res;
     }
 


Test patch:
diff --git a/lib/test/apyfloat/test_rounding.py b/lib/test/apyfloat/test_rounding.py
index 511c7bf9c..7446f4c7b 100644
--- a/lib/test/apyfloat/test_rounding.py
+++ b/lib/test/apyfloat/test_rounding.py
@@ -1164,6 +1164,13 @@ def test_quantization_ties_even(self):
             APyFloat(1, 30, 0, 5, 5).cast(4, 3).is_identical(APyFloat(1, 15, 0, 4, 3))
         )  # Should become infinity
 
+        # From https://github.com/apytypes/apytypes/issues/406
+        res = APyFloat(sign=0, exp=103, man=0, exp_bits=8, man_bits=23).cast(5, 10)
+        assert res.is_identical(APyFloat(sign=0, exp=0, man=1, exp_bits=5, man_bits=10))
+
+        res = APyFloat(sign=0, exp=103, man=1, exp_bits=8, man_bits=23).cast(5, 10)
+        assert res.is_identical(APyFloat(sign=0, exp=0, man=1, exp_bits=5, man_bits=10))
+
     def test_quantization_ties_odd(self):
         apytypes.set_float_quantization_mode(QuantizationMode.TIES_ODD)
         # Quantization from 0.xx
@@ -1945,6 +1952,12 @@ def test_quantization_stochastic_equal(self):
             if done_down and done_up:
                 break
 
+    def test_quantization_subnormal_shift(self):
+        """Result will be subnormal but require a left shift."""
+        assert (_ := APyFloat(0, 0, 1, 4, 4).cast(4, 10)).is_identical(
+            APyFloat(sign=0, exp=0, man=64, exp_bits=4, man_bits=10)
+        )
+
 
 def test_convenience_cast():
     a = APyFloat.from_float(1 / 7, 20, 60)