Repository: cython/cython

Issue:
Overflow error in doctests
I am trying to build Cython 0.27 and 0.27.1 for openSUSE and I am getting an OverFlow error in the docstests when I run `python3 runtests.py -vv` (but not for python2) under an x86_64 architecture.  This works fine up to Cython 0.26.1, but fails for 0.27 and 0.27.1 in the same way.  We are using python 3.6.2 and 2.7.13 and gcc 7.2.1.

```
======================================================================
FAIL: int_abs (builtin_abs)
Doctest: builtin_abs.int_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.int_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in int_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.int_abs
Failed example:
    int_abs(-max_int-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -2147483648


======================================================================
FAIL: long_abs (builtin_abs)
Doctest: builtin_abs.long_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.long_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in long_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.long_abs
Failed example:
    long_abs(-max_long-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -9223372036854775808


======================================================================
FAIL: long_long_abs (builtin_abs)
Doctest: builtin_abs.long_long_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.long_long_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in long_long_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.long_long_abs
Failed example:
    long_long_abs(-max_long_long-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -9223372036854775808


======================================================================
FAIL: int_abs (builtin_abs)
Doctest: builtin_abs.int_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.int_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in int_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.int_abs
Failed example:
    int_abs(-max_int-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -2147483648


======================================================================
FAIL: long_abs (builtin_abs)
Doctest: builtin_abs.long_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.long_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in long_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.long_abs
Failed example:
    long_abs(-max_long-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -9223372036854775808


======================================================================
FAIL: long_long_abs (builtin_abs)
Doctest: builtin_abs.long_long_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.long_long_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in long_long_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.long_long_abs
Failed example:
    long_long_abs(-max_long_long-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -9223372036854775808
```


Discussion:
I was unable to reproduce this on sles-11-sp4-v20170621 with Cython 0.27.1, Python 3.6.2, and gcc `gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)`.
I also tried `SUSE Linux Enterprise Server 12 SP3`, can't get it to fail. 
Also reported in Gentoo https://bugs.gentoo.org/632519 - I can reproduce it with gcc-6.4.0. I am guessing that is because gcc now default to C99 (I think) which wouldn't happen with the compilers you tried.
I don't have this anymore with 0.28. I have another strange issue though that may or may not be related to this.
The problem seems to be caused by `-fstrict-overflow` (implied by `-O2`). Evidence:

```
CFLAGS='-O1' ./runtests.py  builtin_abs  # -> pass
CFLAGS='-O2' ./runtests.py  builtin_abs  # -> fail
CFLAGS='-O2 -fno-strict-overflow' ./runtests.py  builtin_abs  # -> pass
CFLAGS='-O1 -fstrict-overflow' ./runtests.py  builtin_abs  # -> fail
```

This is with gcc-6.4.0. I don't have older versions of gcc to test.
This is still occurring (gcc 9.3.0, `python2-config --cflags` equal to `-I/usr/include/python2.7 -I/usr/include/python2.7 -fno-strict-aliasing -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -DNDEBUG -march=x86-64 -mtune=generic -O2 -pipe -fno-plt`; I think the `O3` is the only relevant part here!).

The code we're producing in all these cases is:

```C
__pyx_t_1 = abs(__pyx_v_a);
if (unlikely(__pyx_t_1 < 0)) {
  PyErr_SetString(PyExc_OverflowError, "value too large");
  __PYX_ERR(0, 60, __pyx_L1_error);
}
```

Calling abs with a value that cannot be converted to an absolute value is undefined behaviour, . it's return value is guaranteed to be non-negative for all defined cases, so resultantly gcc removes the branch as its dead code under C semantics. `-fno-strict-overflow` or `-fwrapv` stop this under gcc, but clang appears to always elide the conditional statement (which is probably a bug!).
Yes, that looks wrong. I think we have to test the input argument upfront, not the result.

The check is generated in `ExprNodes.py`, `SimpleCallNode.generate_result_code()`. I think it should use a dedicated expression node instead, to be inserted into the tree in `analyse_c_function_call()`. The fact that the safety check is currently implemented directly in the code generation of `SimpleCallNode` is rather a hack.

Resolving patch:
diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py
--- a/Cython/Compiler/ExprNodes.py
+++ b/Cython/Compiler/ExprNodes.py
@@ -5934,6 +5934,17 @@ def generate_evaluation_code(self, code):
         if function.is_name or function.is_attribute:
             code.globalstate.use_entry_utility_code(function.entry)
 
+        abs_function_cnames = ('abs', 'labs', '__Pyx_abs_longlong')
+        is_signed_int = self.type.is_int and self.type.signed
+        if self.overflowcheck and is_signed_int and function.result() in abs_function_cnames:
+            code.globalstate.use_utility_code(UtilityCode.load_cached("Common", "Overflow.c"))
+            code.putln('if (unlikely(%s == __PYX_MIN(%s))) {\
+                PyErr_SetString(PyExc_OverflowError,\
+                                "Trying to take the absolute value of the most negative integer is not defined."); %s; }' % (
+                            self.args[0].result(),
+                            self.args[0].type.empty_declaration_code(),
+                            code.error_goto(self.pos)))
+
         if not function.type.is_pyobject or len(self.arg_tuple.args) > 1 or (
                 self.arg_tuple.args and self.arg_tuple.is_literal):
             super(SimpleCallNode, self).generate_evaluation_code(code)
@@ -6036,13 +6047,7 @@ def generate_result_code(self, code):
                                             self.result() if self.type.is_pyobject else None,
                                             func_type.exception_value, self.nogil)
                 else:
-                    if (self.overflowcheck
-                        and self.type.is_int
-                        and self.type.signed
-                        and self.function.result() in ('abs', 'labs', '__Pyx_abs_longlong')):
-                        goto_error = 'if (unlikely(%s < 0)) { PyErr_SetString(PyExc_OverflowError, "value too large"); %s; }' % (
-                            self.result(), code.error_goto(self.pos))
-                    elif exc_checks:
+                    if exc_checks:
                         goto_error = code.error_goto_if(" && ".join(exc_checks), self.pos)
                     else:
                         goto_error = ""


Test patch:
diff --git a/tests/run/builtin_abs.pyx b/tests/run/builtin_abs.pyx
--- a/tests/run/builtin_abs.pyx
+++ b/tests/run/builtin_abs.pyx
@@ -1,5 +1,6 @@
 # mode: run
 # ticket: 698
+# distutils: extra_compile_args=-fwrapv
 
 cdef extern from *:
     int INT_MAX