Issue 1:
Hi,

I've just noticed a dangerous "silent overflow" in Numpy when used in Jupyter notebooks. I understand there were other discussions about similar silent overflows, but this has really put me on the wrong foot, and I believe there must be millions of calculations out there silently failing, and thus giving wrong results.

Here is a function that stretches the values of an array between a given minimum and maximum:

import numpy as np
from scipy import ndimage

def stretcher(a, new_min, new_max):
old_min = ndimage.minimum(a)
old_max = ndimage.maximum(a)
if old_min == old_max:
return a - old_min + new_min
return new_min + (a - old_min) * (new_max - new_min) / (old_max - old_min)

If we execute this code that stretches the values to a positive interval:

a = np.array([0, -182, -10, 0], np.int16)
b = stretcher(a, new_min=np.int16(0), new_max=np.int16(182))
print(b)

we get this result:

[-178.08791209 0. 172. -178.08791209]

A very logic result! But not for earthlings like me... You see, I understand that 0 + 182 * 182 / 182 returns -178.08791209 because it is interpreted as 0 + (182 * 182) / 182 , and 182 * 182 = 33124, which is greater than the maximum that np.int16 can hold (32767). However, I would prefer the execution to fail, so that I know I must take extra measures in my code to catch such overflows.

Moreover, print(b.dtype) returns float64, so one would never think that 182*182 did not fit in a 64 bit float...

The correct result is returned by this slightly modified statement (notice the grouping of the division operation):

return new_min + (a - old_min) * ((new_max - new_min) / (old_max - old_min))
[182. 0. 172. 182.]

The result type is float32 in this case (don't ask me why).

The fact that x*y/z returns a different result than x*(y/z) when z is not zero (nor close to zero) and x, y, and z are quite moderate values is also a concerning issue.

I don't know how Numpy optimizes calculations, but when you see a division (which seems to automatically convert the result to float), you might want to convert all values from that statement to float before any operation takes place.

I see that if I run this:

print(np.int16(182)*np.int16(182))

in a Jupyter notebook I do get the result -32412 and a warning:

[path to Python here]\lib\site-packages\ipykernel\__main__.py:1: RuntimeWarning: overflow encountered in short_scalars
if __name__ == '__main__':

Currently there is no warning in Jupyter notebooks when the same operation is inside a function. It is so easy to believe that everything works fine!

You may say that it's Jupyter's fault, but if the integer overflow would have caused an error, I would have seen the problem immediately in Jupyter, too.

You may also say that I should not work with int16, but with int32 or int64. However, I am constrained by the libraries I work with. Some of them accept only uint8, others int16, and others int32. I already do a lot of conversions back and forth. If I have to convert each time I fear an overflow may happen the code will be messy and slow. I currently cast just before feeding the data to those libraries that accept a single type of data.

By the way, wasn't Python supposed to automatically cast to wider range types when needed? I understand that Numpy tries to keep the type of the result unchanged if the two operands have the same type, but in this case the type will change anyway, so why imposing a constraint (resulting in an error) when the constraint is lifted automatically one step further in the same statement?

I think it is more logical, useful, pythonic and correct (numerically speaking) to automatically cast to higher types when needed and either issuing a warning about such cast, or returning an extra flag indicating that the type has changed, than to return a wrong result and issue a warning about it (which disappears anyway as shown above).

Thanks,

Andrei

Issue 2:
Simple demo:

In [1]: np.uint8(0) - np.uint8(1)
RuntimeWarning: overflow encountered in ubyte_scalars
Out[1]: 255

In [2]: np.uint8(0)[...] - np.uint8(1)[...]
Out[2]: 255

Should we even warn at all for unsigned integers?