Repository: keras-team/keras

Issue:
keras.backend.arange does not allow tensor start, tensorflow.range does
If I try out following script:

```
import tensorflow as tf
from keras import backend as K

K.arange(K.shape(K.constant([1]))[0])
```

I get following error:

```
Traceback (most recent call last):

  File "<ipython-input-30-9bb33f311b39>", line 4, in <module>
    K.arange(K.shape(K.constant([1]))[0])

  File "C:\Anaconda\lib\site-packages\keras\backend\tensorflow_backend.py", line 1871, in arange
    if stop is None and start < 0:

  File "C:\Anaconda\lib\site-packages\tensorflow\python\framework\ops.py", line 564, in __bool__
    raise TypeError("Using a `tf.Tensor` as a Python `bool` is not allowed. "

TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

Traceback (most recent call last):

  File "<ipython-input-30-9bb33f311b39>", line 4, in <module>
    K.arange(K.shape(K.constant([1]))[0])

  File "C:\Anaconda\lib\site-packages\keras\backend\tensorflow_backend.py", line 1871, in arange
    if stop is None and start < 0:

  File "C:\Anaconda\lib\site-packages\tensorflow\python\framework\ops.py", line 564, in __bool__
    raise TypeError("Using a `tf.Tensor` as a Python `bool` is not allowed. "

TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.
```

However if I replace K.arange with tf.range this just works, looking at the source code the implementation for arange seems simple enough:

```
def arange(start, stop=None, step=1, dtype='int32'):
    """Creates a 1D tensor containing a sequence of integers.
    The function arguments use the same convention as
    Theano's arange: if only one argument is provided,
    it is in fact the "stop" argument.
    The default type of the returned tensor is `'int32'` to
    match TensorFlow's default.
    # Arguments
        start: Start value.
        stop: Stop value.
        step: Difference between two successive values.
        dtype: Integer dtype to use.
    # Returns
        An integer tensor.
    """
    # Match the behavior of numpy and Theano by returning an empty seqence.
    if stop is None and start < 0:
        start = 0
    result = tf.range(start, limit=stop, delta=step, name='arange')
    if dtype != 'int32':
        result = cast(result, dtype)
    return result
```

The problem is obviously the `start < 0` part, as `K.shape(x)[0]` outputs a tensor instead of a number, so the < also becomes a tensor.  I don't really see a way around this, but it would be a shame if this simple check would make K.arange less powerful than tf.range.


Discussion:

Resolving patch:
diff --git a/keras/backend/tensorflow_backend.py b/keras/backend/tensorflow_backend.py
--- a/keras/backend/tensorflow_backend.py
+++ b/keras/backend/tensorflow_backend.py
@@ -2055,8 +2055,16 @@ def arange(start, stop=None, step=1, dtype='int32'):
 
     """
     # Match the behavior of numpy and Theano by returning an empty seqence.
-    if stop is None and start < 0:
-        start = 0
+    if stop is None:
+        try:
+            if start < 0:
+                start = 0
+        except TypeError:
+            # Handle case where start is a tensor
+            start = tf.cond(start < 0,
+                            true_fn=lambda: tf.constant(0, dtype=start.dtype),
+                            false_fn=lambda: start)
+
     result = tf.range(start, limit=stop, delta=step, name='arange')
     if dtype != 'int32':
         result = cast(result, dtype)


Test patch:
diff --git a/tests/keras/backend/backend_test.py b/tests/keras/backend/backend_test.py
--- a/tests/keras/backend/backend_test.py
+++ b/tests/keras/backend/backend_test.py
@@ -1371,6 +1371,15 @@ def test_arange(self):
                 t = backend.arange(10, dtype=dtype)
                 assert backend.dtype(t) == dtype
 
+        for backend in [KTH, KTF]:
+            start = backend.constant(1, dtype='int32')
+            t = backend.arange(start)
+            assert len(backend.eval(t)) == 1
+
+            start = backend.constant(-1, dtype='int32')
+            t = backend.arange(start)
+            assert len(backend.eval(t)) == 0
+
     def test_in_train_phase(self):
         for training in [True, False]:
             check_two_tensor_operation('in_train_phase', (3, 3), (2, 2), [KTH, KTF],