Repository: blaze/blaze

Issue:
"ValueError: Not understood" for Mongo after upgrade
I updated blaze today (blaze  {0.6.6_212_g4b7c730 -> 0.8.0}) and I got a new error. 

``` python
import blaze as bz
apps = bz.Data("mongodb://localhost:27017/API::apps", dshape = 'var * {name: string}')
```

That used to run fine but now I get this traceback:

``` python
Traceback (most recent call last):

  File "<ipython-input-1-81a88c5ec220>", line 1, in <module>
    runfile('/Users/Dalton/Desktop/errorTest.py', wdir='/Users/Dalton/Desktop')

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 78, in execfile
    builtins.execfile(filename, *where)

  File "/Users/Dalton/Desktop/errorTest.py", line 10, in <module>
    apps = bz.Data("mongodb://localhost:27017/API::apps", dshape = 'var * {name: string}')

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/blaze/interactive.py", line 100, in Data
    result = result[field]

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/blaze/expr/expressions.py", line 108, in __getitem__
    raise ValueError("Not understood %s[%s]" % (self, key))

ValueError: Not understood _1[apps]
```

Let me know if there's anything I can do to be helpful. 



Discussion:
You likely need to update odo, as there were some changes to the mongo backend. They shouldn't be breaking changes. Can you show your odo version?

I'd recommend using conda to install blaze releases. It'll handle dependencies without any issues. 

Ya I'm using conda. Here's the revisions for the update that preceded the error:

```
conda update blaze
```

```
2015-05-02 14:33:09  (rev 15)
     bcolz  {0.7.2 -> 0.8.1}
     blaze  {0.6.6_212_g4b7c730 -> 0.8.0}
     conda  {3.10.0 -> 3.11.0}
     conda-env  {2.1.3 -> 2.1.4}
     cytoolz  {0.7.0 -> 0.7.2}
     datashape  {0.4.1 -> 0.4.5}
     h5py  {2.3.1 -> 2.5.0}
     hdf5  {1.8.13 -> 1.8.14}
     numba  {0.14.0 -> 0.18.2}
     pandas  {0.15.2 -> 0.16.0}
     pip  {6.0.8 -> 6.1.1}
     psutil  {2.1.1 -> 2.2.1}
     pymongo  {2.7.2 -> 2.8}
     pytables  {3.1.1 -> 3.1.1}
     python-dateutil  {2.4.1 -> 2.4.2}
     pytz  {2014.9 -> 2015.2}
     requests  {2.6.0 -> 2.6.2}
     setuptools  {14.3 -> 15.2}
     sqlalchemy  {0.9.8 -> 1.0.2}
     sqlite  {3.8.4.1 -> 3.8.4.1}
     toolz  {0.7.0 -> 0.7.1}
     werkzeug  {0.9.6 -> 0.10.4}
    +blaze-core-0.8.0
    +enum34-1.0.4
    +funcsigs-0.4
    +llvmlite-0.4.0
    +odo-0.3.2
```

@cpcloud Any other advice on what I might try?

@TDaltonC What happens when you do this?

``` python
import blaze as bz
apps = bz.Data("mongodb://localhost:27017/API::apps")
```

I'm not sure what's going on now but I'm getting an error reporting that there is no blaze module:

``` python
In [4]: import blaze
Traceback (most recent call last):

  File "<ipython-input-4-5c5ee3cb747a>", line 1, in <module>
    import blaze

ImportError: No module named blaze
```

Here's my path:

``` python
In [7]: import sys

In [8]: print sys.path
['', '/Users/Dalton/anaconda/lib/python27.zip',
'/Users/Dalton/anaconda/lib/python2.7',
'/Users/Dalton/anaconda/lib/python2.7/plat-darwin', 
'/Users/Dalton/anaconda/lib/python2.7/plat-mac', 
'/Users/Dalton/anaconda/lib/python2.7/plat-mac/lib-scriptpackages', 
'/Users/Dalton/anaconda/lib/python2.7/lib-tk',
'/Users/Dalton/anaconda/lib/python2.7/lib-old', 
'/Users/Dalton/anaconda/lib/python2.7/lib-dynload', 
'/Users/Dalton/anaconda/lib/python2.7/site-packages/cryptography-0.8-py2.7-macosx-10.5-x86_64.egg', 
'/Users/Dalton/anaconda/lib/python2.7/site-packages/setuptools-15.2-py2.7.egg', 
'/Users/Dalton/anaconda/lib/python2.7/site-packages', 
'/Users/Dalton/anaconda/lib/python2.7/site-packages/PIL', 
'/Users/Dalton/anaconda/lib/python2.7/site-packages/Sphinx-1.2.3-py2.7.egg', 
'/Users/Dalton/anaconda/lib/python2.7/site-packages/aeosa', 
'/Users/Dalton/anaconda/lib/python2.7/site-packages/IPython/extensions', 
'/Users/Dalton/.ipython']
```

and conda says I have blaze:

```
Thomass-MacBook-Pro:~ Dalton$ conda update blaze
Fetching package metadata: ....
# All requested packages already installed.
# packages in environment at /Users/Dalton/anaconda:
#
blaze                     0.8.0                np19py27_0 
```

Every other import I've tried is fine, and they have the same version reported by conda:

``` pandas
scipy.__version__
Out[17]: '0.15.1'

odo.__version__
Out[18]: '0.3.2'

pandas.__version__
Out[19]: '0.16.0'
```

```
# All requested packages already installed.
# packages in environment at /Users/Dalton/anaconda:
#
odo                       0.3.2                np19py27_0  
pandas                    0.16.0               np19py27_1  
scipy                     0.15.1               np19py27_0  
```

I'm going to try to debug this and get back to you about your question, but chime-in if something jumps out at you. 

Make sure you didn't name your file `blaze.py`.

Good though, but nope. No files to folders named 'blaze'. It happens from the terminal too:

```
Thomass-MacBook-Pro:~ Dalton$ python
Python 2.7.9 |Anaconda 2.2.0 (x86_64)| (default, Dec 15 2014, 10:37:34) 
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> import blaze
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named blaze
>>> 
```

try

``` sh
conda remove blaze
pip uninstall blaze
```

and make sure you don't have multiple versions of blaze lying around

If that fails, create new empty conda environment to try and reproduce the error

``` sh
conda create -n testenv python=2 blaze -c blaze
source activate testenv
python -c 'import blaze'
```

Ok, I've got it working now. I think it was that I had multiple versions of blaze. 

Anyway, when I run:

``` python
import blaze as bz

apps = bz.Data("mongodb://localhost:27017/API::apps")
```

I get this traceback:

``` python
In [5]: runfile('/Users/Dalton/Desktop/errorTest.py', wdir='/Users/Dalton/Desktop')
Traceback (most recent call last):

  File "<ipython-input-5-81a88c5ec220>", line 1, in <module>
    runfile('/Users/Dalton/Desktop/errorTest.py', wdir='/Users/Dalton/Desktop')

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 78, in execfile
    builtins.execfile(filename, *where)

  File "/Users/Dalton/Desktop/errorTest.py", line 10, in <module>
    apps = bz.Data("mongodb://localhost:27017/API::apps")

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/blaze/interactive.py", line 67, in Data
    dshape = discover(data)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py", line 163, in __call__
    return func(*args, **kwargs)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/odo/backends/mongo.py", line 111, in discover_mongo_database
    for name in names))))

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/odo/backends/mongo.py", line 111, in <genexpr>
    for name in names))))

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py", line 163, in __call__
    return func(*args, **kwargs)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/odo/backends/mongo.py", line 28, in discover_pymongo_collection
    ds = discover(items)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py", line 163, in __call__
    return func(*args, **kwargs)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/datashape/discovery.py", line 197, in discover
    for column in columns]

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py", line 163, in __call__
    return func(*args, **kwargs)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/datashape/discovery.py", line 342, in discover
    return Record([[k, discover(d[k])] for k in sorted(d)])

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py", line 163, in __call__
    return func(*args, **kwargs)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/datashape/discovery.py", line 342, in discover
    return Record([[k, discover(d[k])] for k in sorted(d)])

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py", line 163, in __call__
    return func(*args, **kwargs)

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/datashape/discovery.py", line 54, in discover
    type(o).__name__)

NotImplementedError: Don't know how to discover type 'ObjectId'
```

Cool I can take a look tomorrow first thing. That's an error I can work with :)

Thanks Phillip. 

In case it helps. This syntax:

``` python
import blaze as bz
db = bz.Data(bz.resource("mongodb://localhost:27017/API::apps"),dshape = 'var * {name: string}')
```

runs fine with blaze=0.8.0

And this syntax:

``` python
import blaze as bz
db = bz.Data(bz.resource("mongodb://localhost:27017/API::apps"))
```

gives this error:

``` python
ValueError: Consistent datashape not found
```

And that is really beyond the scope of this bug. 

@TDaltonC I'm not sure what the error is here anymore. Can you roll back to an earlier version of blaze odo and datashape and show me the result of the following:

``` python
print(bz.Data("mongodb://localhost:27017/API::apps").dshape)
```

Ya. Any version in particular? 

I'd probably go back to the one you reported at the beginning (`0.6.6_212_g4b7c730`)

Hm, on second thought that's a pretty old version. I'm not totally sure what version of odo and datashape will be compatible with that version of blaze.

Have your data changed between the older version of blaze and the newer version?

Nope, but my 'app' collection is a pretty irregularly shaped. That's why I was explicitly providing a dshape before. 

Ah, in that case does the following work for you:

``` python
data = bz.Data("mongodb://localhost:27017/API::apps", dshape='var * {name: string}')
```

?

In blaze=0.8.0 that gives this error

``` python
In [2]: data = bz.Data("mongodb://localhost:27017/API::apps", dshape='var * {name: string}')
Traceback (most recent call last):

  File "<ipython-input-2-47e4941485a3>", line 1, in <module>
    data = bz.Data("mongodb://localhost:27017/API::apps", dshape='var * {name: string}')

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/blaze/interactive.py", line 100, in Data
    result = result[field]

  File "/Users/Dalton/anaconda/lib/python2.7/site-packages/blaze/expr/expressions.py", line 108, in __getitem__
    raise ValueError("Not understood %s[%s]" % (self, key))

ValueError: Not understood _1[apps]
```

but this syntax:

``` python
In [3]: data = bz.Data(bz.resource("mongodb://localhost:27017/API::apps"),dshape = 'var * {name: string}')
```

Runs fine. 

Would you like me to try a different version?

Ah okay. I understand now.

@TDaltonC finally able to reproduce this!

Glad to hear!


Resolving patch:
diff --git a/blaze/interactive.py b/blaze/interactive.py
--- a/blaze/interactive.py
+++ b/blaze/interactive.py
@@ -51,10 +51,7 @@ def Data(data, dshape=None, name=None, fields=None, columns=None, schema=None,
         raise ValueError("Please specify one of schema= or dshape= keyword"
                          " arguments")
 
-    sub_uri = ''
     if isinstance(data, _strtypes):
-        if '::' in data:
-            data, sub_uri = data.split('::')
         data = resource(data, schema=schema, dshape=dshape, columns=columns,
                         **kwargs)
     if (isinstance(data, Iterator) and
@@ -93,14 +90,7 @@ def Data(data, dshape=None, name=None, fields=None, columns=None, schema=None,
             dshape = DataShape(*(dshape.shape + (schema,)))
 
     ds = datashape.dshape(dshape)
-    result = InteractiveSymbol(data, ds, name)
-
-    if sub_uri:
-        for field in sub_uri.split('/'):
-            if field:
-                result = result[field]
-
-    return result
+    return InteractiveSymbol(data, ds, name)
 
 
 class InteractiveSymbol(Symbol):
diff --git a/blaze/server/__init__.py b/blaze/server/__init__.py
--- a/blaze/server/__init__.py
+++ b/blaze/server/__init__.py
@@ -2,7 +2,7 @@
 
 from .server import Server, to_tree, from_tree, api
 from .spider import spider, from_yaml
-from .client import ExprClient, Client
+from .client import Client
 from .serialization import (
     SerializationFormat,
     all_formats,
@@ -14,7 +14,6 @@
 
 __all__ = [
     'Client',
-    'ExprClient',
     'SerializationFormat',
     'Server',
     'spider',
diff --git a/blaze/server/client.py b/blaze/server/client.py
--- a/blaze/server/client.py
+++ b/blaze/server/client.py
@@ -16,7 +16,7 @@
 # flask for testing.  Sadly they have different Response objects,
 # hence the dispatched functions
 
-__all__ = 'Client', 'ExprClient'
+__all__ = 'Client',
 
 
 def content(response):
@@ -90,12 +90,6 @@ def dshape(self):
         return dshape(content(response).decode('utf-8'))
 
 
-def ExprClient(*args, **kwargs):
-    import warnings
-    warnings.warn("Deprecated use `Client` instead", DeprecationWarning)
-    return Client(*args, **kwargs)
-
-
 @dispatch(Client)
 def discover(c):
     return c.dshape
@@ -116,11 +110,17 @@ def compute_down(expr, ec, **kwargs):
 
 
 @resource.register('blaze://.+')
-def resource_blaze(uri, **kwargs):
+def resource_blaze(uri, leaf=None, **kwargs):
+    if leaf is not None:
+        raise ValueError('The syntax blaze://...::{leaf} is no longer '
+                         'supported as of version 0.8.1.\n'
+                         'You can access {leaf!r} using this syntax:\n'
+                         'Data({uri})[{leaf!r}]'
+                         .format(leaf=leaf, uri=uri))
     uri = uri[len('blaze://'):]
     sp = uri.split('/')
     tld, rest = sp[0], sp[1:]
     if ':' not in tld:
-        tld = tld + ':%d' % DEFAULT_PORT
+        tld += ':%d' % DEFAULT_PORT
     uri = '/'.join([tld] + list(rest))
     return Client(uri)


Test patch:
diff --git a/blaze/compute/tests/test_mongo_compute.py b/blaze/compute/tests/test_mongo_compute.py
--- a/blaze/compute/tests/test_mongo_compute.py
+++ b/blaze/compute/tests/test_mongo_compute.py
@@ -368,3 +368,9 @@ def test_and_same_key(bank):
     result = compute(expr, bank)
     expected = [('Alice', 200), ('Bob', 200)]
     assert result == expected
+
+
+def test_interactive_dshape_works():
+    d = Data('mongodb://localhost:27017/test_db::bank',
+             dshape='var * {name: string, amount: int64}')
+    assert d.dshape == dshape('var * {name: string, amount: int64}')
diff --git a/blaze/server/tests/test_client.py b/blaze/server/tests/test_client.py
--- a/blaze/server/tests/test_client.py
+++ b/blaze/server/tests/test_client.py
@@ -99,6 +99,13 @@ def test_custom_expressions():
     assert list(map(tuple, compute(CustomExpr(t.accounts), ec))) == into(list, df)
 
 
+def test_client_dataset_fails():
+    with pytest.raises(ValueError):
+        Data('blaze://localhost::accounts')
+    with pytest.raises(ValueError):
+        resource('blaze://localhost::accounts')
+
+
 def test_client_dataset():
-    d = Data('blaze://localhost::accounts')
-    assert list(map(tuple, into(list, d))) == into(list, df)
+    d = Data('blaze://localhost')
+    assert list(map(tuple, into(list, d.accounts))) == into(list, df)