LowinData/pyautodiff — pull request #7 by gwtaylor (2014-09-20 04:52:30+00:00)
Title: Fix 'The truth value of an array' error

Thanks for fixing the bug mentioned in Issue #5. Extending the example there, I uncovered another problem. When running the following test program

```python
# some basic imports we need for this tutorial
import numpy as np
from autodiff.optimize import fmin_l_bfgs_b

eps = np.finfo(np.double).eps # -- a small number

dtype = 'float32'

# some fake data for this example
x = np.random.randn(100, 784)
y = np.random.randint(2, size=x.shape[0])

def logistic(u):
"""Return logistic sigmoid of float or ndarray `u`"""
return 1.0 / (1.0 + np.exp(-u))

def p_y_given_x(W, b, x):
return logistic(np.dot(x, W) + b)

def logreg_prediction(W, b, x):
return p_y_given_x(W, b, x) > 0.5

def cross_entropy(x, z):
# note we add a small epsilon for numerical stability
return -(x * np.log(z + eps) + (1 - x) * np.log(1 - z + eps))

def logreg_cost(W, b, x, y):
z = p_y_given_x(W, b, x)
l = cross_entropy(y, z).mean(axis=0)
return l

def tanh_layer(V, c, x):
return np.tanh(np.dot(x, V) + c)

def mlp_prediction(V, c, W, b, x):
h = tanh_layer(V, c, x)
return logreg_prediction(W, b, h)

def mlp_cost(V, c, W, b, x, y):
h = tanh_layer(V, c, x)
return logreg_cost(W, b, h, y)

batch_criterion = lambda V, c, W, b: mlp_cost(V, c, W, b, x, y)

# initialize the model
n_hidden = 100
V = np.zeros((x.shape[1], n_hidden), dtype=dtype)
c = np.zeros(n_hidden, dtype=dtype)
W = np.zeros(n_hidden, dtype=dtype)

# autodiff relies on the .size and .shape
# attributes, which native python types do not have
# need to create a numpy scalar
# note this is different than np.array([0.0])
# which will not broadcast
b = np.array(0.0)

V_new, c_new, W_new, b_new = fmin_l_bfgs_b(batch_criterion, (V, c, W, b),
maxfun=100, iprint=1)
```

I get the following error

```
Traceback (most recent call last):
File "testpyautodiffmlp.py", line 68, in
V_new, c_new, W_new, b_new = fmin_l_bfgs_b(batch_criterion, (V, c, W, b), maxfun=100, iprint=1)
File "/Users/gwtaylor/src/pyautodiff/autodiff/optimize.py", line 111, in fmin_l_bfgs_b
gradient=True)
File "/Users/gwtaylor/src/pyautodiff/autodiff/symbolic.py", line 498, in __init__
_, (sym_vector, result) = symbolic.trace(*init_args, **init_kwargs)
File "/Users/gwtaylor/src/pyautodiff/autodiff/symbolic.py", line 99, in trace
results = self.symfn(*c_args, **c_kwargs)
File "", line 489, in wrapper
File "", line 525, in args_from_vector
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

It's happening on symbolic.py line 525, where ```if arg.shape``` is called on a Theano Tensor object. Strangely, in a python shell, if I create a 2-d Theano tensor and call ``if arg.shape`` on it, I don't get an error. I've made the following fix, which makes my example work, and I think it does what you expect, but I am not really familiar with the code base, so please do verify.