Repository: numpy/numpy

Issue:
array_split incorrect behavior with array size bigger MAX_INT32
`numpy.array_split()` produces empty last chunk in case of original array size is bigger than MAX_INT32.

### Reproducing code example:

**NOT OK**:
```
import numpy as np
test = np.random.rand(1 << 31, 1)
for chunk in np.array_split(test, 2):
    print(chunk.shape)

>>>(1073741824, 1)
>>>(0, 1)
```

**NOT OK**:
```
import numpy as np
test = np.random.rand(1 << 31, 1)
for chunk in np.array_split(test, 3):
    print(chunk.shape)

>>>(715827883, 1)
>>>(715827883, 1)
>>>(0, 1)
```

**OK**:
```
import numpy as np
test = np.random.rand(1 << 30, 1)
for chunk in np.array_split(test, 2):
    print(chunk.shape)

>>>(536870912, 1)
>>>(536870912, 1)
```

**OK**:
```
import numpy as np
test = np.random.rand((1 << 31) - 1, 1)
for chunk in np.array_split(test, 2):
    print(chunk.shape)

>>>(1073741824, 1)
>>>(1073741823, 1)
```

### Numpy/Python version information:

```
numpy.__version__
>>> 1.14.4

sys.version
>>> 3.5.2 |Continuum Analytics, Inc.| (default, Jul  5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)]
```

112 GB of RAM to exclude the problem with the memory lack.


Discussion:
In addition, there is a misleading comment (L687) in the source code of `array_split`:

https://github.com/numpy/numpy/blob/5adb81051086a45fe3b59ba506b567340ec9bd5e/numpy/lib/shape_base.py#L686-L692

Should be:

```
# handle array case.
```
Want to create a PR about it? I am pretty sure the bug is just below those lines of code. There is an array being created and then `cumsum`ed, that array has to be of `dtype=np.intp` (or a similar solution).

Tests should be possible using broadcasted arrays (on 64bit)
@seberg 
Thank you very much for the solution!
I applied your fix locally and it solved the problem of the last empty chunk.
I'll create a PR.

Resolving patch:
diff --git a/numpy/lib/shape_base.py b/numpy/lib/shape_base.py
--- a/numpy/lib/shape_base.py
+++ b/numpy/lib/shape_base.py
@@ -688,7 +688,7 @@ def array_split(ary, indices_or_sections, axis=0):
     except AttributeError:
         Ntotal = len(ary)
     try:
-        # handle scalar case.
+        # handle array case.
         Nsections = len(indices_or_sections) + 1
         div_points = [0] + list(indices_or_sections) + [Ntotal]
     except TypeError:
@@ -700,7 +700,7 @@ def array_split(ary, indices_or_sections, axis=0):
         section_sizes = ([0] +
                          extras * [Neach_section+1] +
                          (Nsections-extras) * [Neach_section])
-        div_points = _nx.array(section_sizes).cumsum()
+        div_points = _nx.array(section_sizes, dtype=_nx.intp).cumsum()
 
     sub_arys = []
     sary = _nx.swapaxes(ary, axis, 0)


Test patch:
diff --git a/numpy/lib/tests/test_shape_base.py b/numpy/lib/tests/test_shape_base.py
--- a/numpy/lib/tests/test_shape_base.py
+++ b/numpy/lib/tests/test_shape_base.py
@@ -3,6 +3,8 @@
 import numpy as np
 import warnings
 import functools
+import sys
+import pytest
 
 from numpy.lib.shape_base import (
     apply_along_axis, apply_over_axes, array_split, split, hsplit, dsplit,
@@ -14,6 +16,9 @@
     )
 
 
+IS_64BIT = sys.maxsize > 2**32
+
+
 def _add_keepdims(func):
     """ hack in keepdims behavior into a function taking an axis """
     @functools.wraps(func)
@@ -403,6 +408,15 @@ def test_integer_split_2D_default(self):
         assert_(a.dtype.type is res[-1].dtype.type)
         # perhaps should check higher dimensions
 
+    @pytest.mark.skipif(not IS_64BIT, reason="Needs 64bit platform")
+    def test_integer_split_2D_rows_greater_max_int32(self):
+        a = np.broadcast_to([0], (1 << 32, 2))
+        res = array_split(a, 4)
+        chunk = np.broadcast_to([0], (1 << 30, 2))
+        tgt = [chunk] * 4
+        for i in range(len(tgt)):
+            assert_equal(res[i].shape, tgt[i].shape)
+
     def test_index_split_simple(self):
         a = np.arange(10)
         indices = [1, 5, 7]