Repository: dask/dask

Issue:
BUG: bcolz dataframes dropping columns after groupby
Hello,

When I run `dask=0.8.1` with `bcolz=1.0.0`, I can get this...

``` python
Python 3.5.1 |Anaconda custom (64-bit)| (default, Feb 16 2016, 09:49:46) [MSC v.1900 64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.

IPython 4.2.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
%guiref   -> A brief reference about the graphical user interface.

In [1]: import dask
   ...: import dask.dataframe as ddf
   ...: import pandas as pd
   ...: import bcolz
   ...: 
   ...: data = pd.DataFrame({'a': ['a', 'a', 'b', 'b', 'c'],
   ...:                      'b': [1, 2, 1, 3, 4],
   ...:                   'c': ['z', 'x', 'x', 'z', 'x']})
   ...: 
   ...: data = bcolz.ctable.fromdataframe(data)
   ...: 
   ...: data = ddf.from_bcolz(data, chunksize=2)
   ...: 
   ...: analysis = data[['a']]
   ...: analysis = analysis.assign(Two=data['b'], One=1)
   ...: analysis.groupby('a').sum().compute()
Out[1]: 
      One  Two
a             
b'a'  2.0  3.0
b'b'  2.0  4.0
b'c'  1.0  4.0

In [2]: bcolz.__version__
Out[2]: '1.0.0'

In [3]: dask.__version__
Out[3]: '0.8.1'
```

When I upgrade to `dask=0.9.0`, the column `Two` drops. (This occurs on `bcolz` only... `pandas` seems to work. Haven't dried any other formats).

``` python
Python 3.5.1 |Anaconda custom (64-bit)| (default, Feb 16 2016, 09:49:46) [MSC v.1900 64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.

IPython 4.2.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
%guiref   -> A brief reference about the graphical user interface.

import dask
import dask.dataframe as ddf
import pandas as pd
import bcolz

data = pd.DataFrame({'a': ['a', 'a', 'b', 'b', 'c'],
                     'b': [1, 2, 1, 3, 4],
                  'c': ['z', 'x', 'x', 'z', 'x']})

data = bcolz.ctable.fromdataframe(data)

data = ddf.from_bcolz(data, chunksize=2)

analysis = data[['a']]
analysis = analysis.assign(Two=data['b'], One=1)
analysis.groupby('a').sum().compute()
Out[1]: 
      One
a        
b'a'  2.0
b'b'  2.0
b'c'  1.0

bcolz.__version__
Out[2]: '1.0.0'

dask.__version__

Out[3]: '0.9.0'
```



Discussion:
The issue was an improperly inferred type. Fix coming soon.


Resolving patch:
diff --git a/dask/dataframe/io.py b/dask/dataframe/io.py
--- a/dask/dataframe/io.py
+++ b/dask/dataframe/io.py
@@ -237,7 +237,8 @@ def from_bcolz(x, chunksize=None, categorize=True, index=None, lock=lock,
                  columns, categories, lock))
                for i in range(0, int(ceil(len(x) / chunksize))))
 
-    result = DataFrame(dsk, new_name, columns, divisions)
+    meta = dataframe_from_ctable(x, slice(0, 0), columns, categories, lock)
+    result = DataFrame(dsk, new_name, meta, divisions)
 
     if index:
         assert index in x.names


Test patch:
diff --git a/dask/dataframe/tests/test_io.py b/dask/dataframe/tests/test_io.py
--- a/dask/dataframe/tests/test_io.py
+++ b/dask/dataframe/tests/test_io.py
@@ -269,6 +269,7 @@ def test_from_bcolz():
     t = bcolz.ctable([[1, 2, 3], [1., 2., 3.], ['a', 'b', 'a']],
                      names=['x', 'y', 'a'])
     d = dd.from_bcolz(t, chunksize=2)
+    assert d._known_dtype
     assert d.npartitions == 2
     assert str(d.dtypes['a']) == 'category'
     assert list(d.x.compute(get=get_sync)) == [1, 2, 3]