Repository: blaze/blaze

Issue:
resource is broken for *.x.y style glob patterns
``` python
In [7]: !touch a.csv.gz

In [8]: !touch b.csv.gz

In [9]: !touch c.csv.gz

In [10]: from blaze import resource

In [11]: resource('*.csv.gz')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-11-85232187192f> in <module>()
----> 1 resource('*.csv.gz')

/Users/pcloud/miniconda/envs/py34/lib/python3.4/site-packages/blaze/regex.py in __call__(self, s, *args, **kwargs)
     60
     61     def __call__(self, s, *args, **kwargs):
---> 62         return self.dispatch(s)(s, *args, **kwargs)

/Users/pcloud/miniconda/envs/py34/lib/python3.4/site-packages/blaze/data/csv.py in resource_csv_gz(uri, **kwargs)
    485 @resource.register('.+\.(\wsv|data|txt|
```

might be a priority issue



Discussion:
This issue was added recently:

@resource.register('.+*._', priority=14)
def resource_glob(uri, *_kwargs):

Should be

@resource.register('._\_._', priority=14)
def resource_glob(uri, *_kwargs):

On Mon, Oct 13, 2014 at 12:44 PM, Phillip Cloud notifications@github.com
wrote:

> In [7]: !touch a.csv.gz
> In [8]: !touch b.csv.gz
> In [9]: !touch c.csv.gz
> In [10]: from blaze import resource
> In [11]: resource('_.csv.gz')---------------------------------------------------------------------------ValueError                                Traceback (most recent call last)<ipython-input-11-85232187192f> in <module>()----> 1 resource('_.csv.gz')
> /Users/pcloud/miniconda/envs/py34/lib/python3.4/site-packages/blaze/regex.py in **call**(self, s, _args, *_kwargs)
>      60
>      61     def **call**(self, s, _args, *_kwargs):---> 62         return self.dispatch(s)(s, _args, *_kwargs)
> /Users/pcloud/miniconda/envs/py34/lib/python3.4/site-packages/blaze/data/csv.py in resource_csv_gz(uri, **kwargs)
>     485 @resource.register('.+.(\wsv|data|txt|
> 
> might be a priority issue
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/ContinuumIO/blaze/issues/738.

shouldn't it be failing? i'm almost certain we test this in some way

i suppose it could have a very generic xfail on it

This shouldn't fail, I think it's acceptable behavior.  Our tests on globstrings look like `account_*.csv` so they pass.  We don't explicitly test the `*.csv` case

sorry i'm a little unclear on what you mean here? is this not a bug or should we go back to the previous regex? i'm +1 on the latter

Yes, sorry, this is a bug.  I'm just trying to help localize the bug and
suggest a fix.

On Mon, Oct 13, 2014 at 3:41 PM, Phillip Cloud notifications@github.com
wrote:

> sorry i'm a little unclear on what you mean here? is this not a bug or
> should we go back to the previous regex? i'm +1 on the latter
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/ContinuumIO/blaze/issues/738#issuecomment-58965587.


Resolving patch:
diff --git a/blaze/__init__.py b/blaze/__init__.py
--- a/blaze/__init__.py
+++ b/blaze/__init__.py
@@ -14,6 +14,7 @@
 from .expr.functions import *
 from .api import *
 from .data import *
+from .json import *
 from .resource import *
 from .compute.dynd import *
 from .compute.python import *
diff --git a/blaze/compute/chunks.py b/blaze/compute/chunks.py
--- a/blaze/compute/chunks.py
+++ b/blaze/compute/chunks.py
@@ -296,7 +296,7 @@ def __iter__(self):
 from ..resource import resource
 from glob import glob
 
-@resource.register('.+\*.*', priority=14)
+@resource.register('.*\*.*', priority=14)
 def resource_glob(uri, **kwargs):
     uris = sorted(glob(uri))
 
diff --git a/blaze/json.py b/blaze/json.py
new file mode 100644
--- /dev/null
+++ b/blaze/json.py
@@ -0,0 +1,26 @@
+from __future__ import absolute_import, division, print_function
+
+import json
+from toolz import map, partial
+import gzip
+
+from .resource import resource
+
+__all__ = 'resource',
+
+@resource.register('.*\.json')
+def resource_json(uri, open=open):
+    f = open(uri)
+    try:
+        data = json.load(f)
+        f.close()
+        return data
+    except ValueError:
+        f = open(uri)
+        data = map(json.loads, f)
+        return data
+
+
+@resource.register('.*\.json.gz')
+def resource_json_gzip(uri):
+    return resource_json(uri, open=partial(gzip.open, mode='rt'))


Test patch:
diff --git a/blaze/tests/test_json.py b/blaze/tests/test_json.py
new file mode 100644
--- /dev/null
+++ b/blaze/tests/test_json.py
@@ -0,0 +1,16 @@
+from __future__ import absolute_import, division, print_function
+
+from blaze.json import resource
+from blaze.utils import example
+
+def test_simple():
+    assert len(list(resource(example('accounts.json')))) == 5
+
+def test_streaming():
+    assert len(list(resource(example('accounts-streaming.json')))) == 5
+
+def test_simple_gzip():
+    assert len(list(resource(example('accounts.json.gz')))) == 5
+
+def test_streaming_gzip():
+    assert len(list(resource(example('accounts-streaming.json.gz')))) == 5