Repository: numpy/numpy

Issue:
Indicate attempted allocation size when constructing an array gives a MemoryError
I've seen multiple questions now of the form "why does this give a MemoryError" when the reason is the user ended up allocating a bigger array than expected.

It would be helpful to include the shape and dtype in the error message.


Discussion:
I will give this a try sometimes this week. If someone else is already working on it let me know. 
For some reason this does not reproduce as reported on my setup (OSx, python 3.7.3): 

```
>>> import numpy as np
>>> t = np.zeros((51,765,285),dtype=np.float64)
>>> z = np.zeros((51,765,285),dtype=np.float64)
>>> indices = np.apply_along_axis(lambda a: a.searchsorted(10), axis=0, arr=t)
>>> indices.shape
(765, 285)
>>> np.take(z, indices, axis=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bma/numpy/numpy/core/overrides.py", line 150, in public_api
    implementation, public_api, relevant_args, args, kwargs)
  File "/Users/bma/numpy/numpy/core/fromnumeric.py", line 192, in take
    return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
  File "/Users/bma/numpy/numpy/core/fromnumeric.py", line 60, in _wrapfunc
    return bound(*args, **kwds)
IndexError: index 51 is out of bounds for size 51
>>> z[indices,:,:]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: index 51 is out of bounds for axis 0 with size 51
```

On the other hand, a fresh install of numpy from git on Elementary OS reproduced it (both setups run 1.17.0). 
I'll keep looking.
@bmakos: You could try testing with `np.ones((2**40,), np.uint8)`, which should be easily greater than your available ram
<img width="1050" alt="numpy extensive memory use" src="https://user-images.githubusercontent.com/5707813/55749327-f10bcd00-5a40-11e9-927c-1e3b21739404.png">

Ok, so it seems I will not be able to reproduce this on my machine, the process gets automatically killed at around 120gb by the os. I'll just use a vm with Elemenary for this issue. Sry for the confusion. 
I am a bit insecure about the solution I took. Is it acceptable to issue just this amount of information for this error? 
These lines are where the change would go:

https://github.com/numpy/numpy/blob/f4a24b1f4779d1b9966efe2ad79cba21241ce500/numpy/core/src/multiarray/ctors.c#L1054-L1080
Yes, I was already working on adding it. I debugged it with gdb last night and I got to the same conclusion: 

```
Breakpoint 1, 0x0000555555602337 in PyErr_NoMemory ()
(gdb) up
#1  0x00007ffff5b3b131 in PyArray_NewFromDescr_int (subtype=0x7ffff60b9d00 <PyArray_Type>,
    descr=0x7ffff60ba660 <DOUBLE_Descr>, nd=4, dims=0x7fffffffcb30, strides=0x0, data=0x0, flags=0,
    obj=0x7ffff6163ad0, base=0x0, zeroed=0, allow_emptystring=0)
    at numpy/core/src/multiarray/ctors.c:1075
1075                PyErr_NoMemory();
(gdb)
```
I'll use `PyErr_SetObject` or `PyErr_SetString`, I just have to figure out how to build the message. 

@eric-wieser Thank you for confirming the location, it's good to know I am on track. 
One trick you could use is to dispatch the message formatting to python. One approach would be to add a helper function to `_internal.py`, the other would be to create a MemoryError subclass in `_exceptions.py`. search the c code for the pyhon function names found in those files.

Resolving patch:
diff --git a/numpy/core/_exceptions.py b/numpy/core/_exceptions.py
--- a/numpy/core/_exceptions.py
+++ b/numpy/core/_exceptions.py
@@ -121,3 +121,15 @@ def __init__(self, axis, ndim=None, msg_prefix=None):
                 msg = "{}: {}".format(msg_prefix, msg)
 
         super(AxisError, self).__init__(msg)
+
+
+@_display_as_base
+class _ArrayMemoryError(MemoryError):
+    """ Thrown when an array cannot be allocated"""
+    def __init__(self, shape, dtype):
+        self.shape = shape
+        self.dtype = dtype
+
+    def __str__(self):
+        return "Unable to allocate array with shape {} and data type {}".format(self.shape, self.dtype)
+


Test patch: