Repository: numpy/numpy

Issue:
Fixed bug in numpy.piecewise() for 0-d array handling
Updated numpy/lib/function_base.py to fix bug in numpy.piecewise() for 0-d array handling and boolean indexing with scalars.

Bug test case:

```
>>> numpy.piecewise(5, [True, False], [1, 0])
Traceback (most recent call last):
    [...]
    y[condlist[k]] = item
ValueError: boolean index array has too many values
```

After fix:

```
>>> numpy.piecewise(5, [True, False], [1, 0])
array(1)
```



Discussion:
This pull request [fails](http://travis-ci.org/numpy/numpy/builds/1788960) (merged e553e1b4 into 731cf3aa).

The current version of this pull request appears to fail the existing tests (e.g., this build reports failures: http://travis-ci.org/#!/numpy/numpy/jobs/1788964), so that would need to be fixed.

Also, some sort of test needs to be added to make sure that the bug is fixed (and stays that way).

Hi njsmith, thanks. I've fixed both my change to numpy.piecewise() as well as the test cases to cover both my bug case above as well as making sure that the tests are all logical.

This pull request [passes](http://travis-ci.org/numpy/numpy/builds/1804356) (merged e2ad1a72 into 731cf3aa).

I guess you rearranged the order of all the tests in the source? It makes the diff unreadable unfortunately -- I can't tell at all what you've actually changed without reverse-engineering it by hand, and I'm afraid I'm too lazy for that. Can you revert the no-op changes, or split them into a separate commit, or at least describe what changes were actually made?

Is this issue (i.e. this pull request) going to be fixed anytime soon? It's a bit anoying to construct a 1d array with one element each time I want to call this function with a scalar.

I guess it will be fixed as soon as someone does the work :-)

Hm, got it :) Should an issue be filed first?

The main point of filing an issue is to make sure that something doesn't get forgotten, and this PR is already sort of doing that (i.e., it shows up on the list of open issues if anyone goes to look at it). But feel free to file one if you like. If you're thinking about fixing the bug yourself, then we're pretty laid back about these things -- you can just go ahead and submit a new PR without filing an issue if you want. We'd rather get the fix than worry about the process :-).

The main thing that needs done is just to rewrite the unit tests. In my own testing and usage (and when I've tested with the original unit tests), the code works fine, but I just haven't had the time to sit down and rewrite them to be more similar to the original set and still be complete and systematic. I may get to it next week sometime, but until then I'm swamped.

-- Eric

On Nov 15, 2012, at 9:33 AM, njsmith notifications@github.com wrote:

> The main point of filing an issue is to make sure that something doesn't get forgotten, and this PR is already sort of doing that (i.e., it shows up on the list of open issues if anyone goes to look at it). But feel free to file one if you like. If you're thinking about fixing the bug yourself, then we're pretty laid back about these things -- you can just go ahead and submit a new PR without filing an issue if you want. We'd rather get the fix than worry about the process :-).
> 
> —
> Reply to this email directly or view it on GitHub.

That's one of the things I was about to ask, it is ok to rewrite the tests? Because right now IMHO they don't seem very meaningful for the purpose of the function. I'd like to try it myself this weekend, or next one as much.

It's OK to make any changes whatsoever, the requirements are just that you
should have a reason, and we should be able to look at your changes and
figure out if they're good :-). How exactly to do that depends on the
change. Easily readable diffs, English explanations of what you did and
why... whatever works.
On 17 Nov 2012 00:02, "Juan Luis Cano Rodríguez" notifications@github.com
wrote:

> That's one of the things I was about to ask, it is ok to rewrite the
> tests? Because right now IMHO they don't seem very meaningful for the
> purpose of the function. I'd like to try it myself this weekend, or next
> one as much.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/numpy/numpy/pull/331#issuecomment-10467386.

What is the status of this?

Well actually I didn't do it any of the weekends as you can notice :( But anyway now that you ask about it, if I recall correctly there was some discussion recently about rank-0 arrays in the mailing list but I am not able to find it. Are there any constraints on how to do this? Other than that it would be my first contribution to NumPy and I am a bit unsecure but I am willing to give it a try.

Yes, I was referring to this:

http://www.mail-archive.com/numpy-discussion@scipy.org/msg39968.html

My question was if there are any near future plans for rank-0 arrays that relate this and other issues in such a way that it requires changing low-level code or if we can address this individually.

No, I don't think there will be any near-future changes in how 0d arrays
work that will affect this bug.
On 1 Mar 2013 07:29, "Juan Luis Cano Rodríguez" notifications@github.com
wrote:

> Yes, I was referring to this:
> 
> http://www.mail-archive.com/numpy-discussion@scipy.org/msg39968.html
> 
> My question was if there are any near future plans for rank-0 arrays that
> relate this and other issues in such a way that it requires changing
> low-level code or if we can address this individually.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/numpy/numpy/pull/331#issuecomment-14276274
> .

Sorry for the delay of months. Finishing up my degree and job hunting, so I've been...occupied. I reverted most of the unit test changes so that it's clearer what I changed. Maybe I'll re-up the unit test changes at some point, but this particular commit should be easier to understand when you diff between this state and the state 3 commits ago.

@njsmith Can this go in.

Ping Travisbot just because it was a long time ago.

@ericsuh Made some comments. The commits should also be squashed into one.

I've put up a cleaned version at https://github.com/charris/numpy in the branch gh-331

Needs finishing up.

I seem to have deleted my cleanup somewhere along the line.

In the docstring:

"Each boolean array in `condlist` selects a piece of `x`, and should therefore be of the same shape as `x`."

But actually even the tests do not respect this (and still everything works):

https://github.com/numpy/numpy/blob/b1c69df01b673cc086065112da6780d8548a0dfa/numpy/lib/tests/test_function_base.py#L1468

There are other bugs too. For instance, there's `test_0d` but even though `np.piecewise(x, x > 3, [4, 0])` passes, `np.piecewise(x, [x > 3, x <= 3], [4, 0])` fails (same error as originally reported). I don't think this behaviour is consistent - that's what I actually meant a year and a half ago. The precondition stated in the docstring makes sense for me and should be somehow enforced - is anybody against it? I cannot promise anything but I may try and solve this at once.


Resolving patch:
diff --git a/numpy/lib/function_base.py b/numpy/lib/function_base.py
--- a/numpy/lib/function_base.py
+++ b/numpy/lib/function_base.py
@@ -651,7 +651,7 @@ def piecewise(x, condlist, funclist, *args, **kw):
         The output is the same shape and type as x and is found by
         calling the functions in `funclist` on the appropriate portions of `x`,
         as defined by the boolean arrays in `condlist`.  Portions not covered
-        by any condition have undefined values.
+        by any condition have a default value of 0.
 
 
     See Also
@@ -693,32 +693,24 @@ def piecewise(x, condlist, funclist, *args, **kw):
     if (isscalar(condlist) or not (isinstance(condlist[0], list) or
                                    isinstance(condlist[0], ndarray))):
         condlist = [condlist]
-    condlist = [asarray(c, dtype=bool) for c in condlist]
+    condlist = array(condlist, dtype=bool)
     n = len(condlist)
-    if n == n2 - 1:  # compute the "otherwise" condition.
-        totlist = condlist[0]
-        for k in range(1, n):
-            totlist |= condlist[k]
-        condlist.append(~totlist)
-        n += 1
-    if (n != n2):
-        raise ValueError(
-            "function list and condition list must be the same")
-    zerod = False
     # This is a hack to work around problems with NumPy's
     #  handling of 0-d arrays and boolean indexing with
     #  numpy.bool_ scalars
+    zerod = False
     if x.ndim == 0:
         x = x[None]
         zerod = True
-        newcondlist = []
-        for k in range(n):
-            if condlist[k].ndim == 0:
-                condition = condlist[k][None]
-            else:
-                condition = condlist[k]
-            newcondlist.append(condition)
-        condlist = newcondlist
+        if condlist.shape[-1] != 1:
+            condlist = condlist.T
+    if n == n2 - 1:  # compute the "otherwise" condition.
+        totlist = np.logical_or.reduce(condlist, axis=0)
+        condlist = np.vstack([condlist, ~totlist])
+        n += 1
+    if (n != n2):
+        raise ValueError(
+                "function list and condition list must be the same")
 
     y = zeros(x.shape, x.dtype)
     for k in range(n):


Test patch:
diff --git a/numpy/lib/tests/test_function_base.py b/numpy/lib/tests/test_function_base.py
--- a/numpy/lib/tests/test_function_base.py
+++ b/numpy/lib/tests/test_function_base.py
@@ -1487,6 +1487,7 @@ def test_simple(self):
         x = piecewise([0, 0], [[False, True]], [lambda x:-1])
         assert_array_equal(x, [0, -1])
 
+    def test_two_conditions(self):
         x = piecewise([1, 2], [[True, False], [False, True]], [3, 4])
         assert_array_equal(x, [3, 4])
 
@@ -1505,6 +1506,15 @@ def test_0d(self):
         assert_(y.ndim == 0)
         assert_(y == 0)
 
+        x = 5
+        y = piecewise(x, [[True], [False]], [1, 0])
+        assert_(y.ndim == 0)
+        assert_(y == 1)
+
+    def test_0d_comparison(self):
+        x = 3
+        y = piecewise(x, [x <= 3, x > 3], [4, 0])
+
 
 class TestBincount(TestCase):
     def test_simple(self):