Repository: blaze/blaze

Issue:
Bug converting expr with np.asarray
Expressions created seem to no be correctly converted using `np.array`. See Out[113] of the following code:

``` Python
In[82]: vs = {'first': [2., 5., 3.], 'second': [4., 1., 4.], 'third': [6., 4., 3.]}
In[83]: vdf = pd.DataFrame(vs)
In[109]: v1 = blaze.Data(vdf, fields=["A", "B", "C"])
In[110]: v1
Out[110]: 
   first  second  third
0      2       4      6
1      5       1      4
2      3       4      3
In[111]: v2 = blaze.Data(vdf)
In[112]: v2
Out[112]: 
   first  second  third
0      2       4      6
1      5       1      4
2      3       4      3
In[113]: np.asarray(v1)
Out[113]: 
array([(0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)], 
      dtype=[('A', '<f8'), ('B', '<f8'), ('C', '<f8')])
In[114]: np.asarray(v2)
Out[114]: 
array([(2.0, 4.0, 6.0), (5.0, 1.0, 4.0), (3.0, 4.0, 3.0)], 
      dtype=[('first', '<f8'), ('second', '<f8'), ('third', '<f8')])
In[115]: blaze.into(pd.DataFrame, v1)
Out[115]: 
   first  second  third
0      2       4      6
1      5       1      4
2      3       4      3
In[116]: blaze.into(pd.DataFrame, v2)
Out[116]: 
   first  second  third
0      2       4      6
1      5       1      4
2      3       4      3
```



Discussion:
Thinking a bit more about that, I may be misunderstanding the `fields` argument, how it's supposed to be used and the difference with `columns`

Columns is the deprecated version of fields. They do the same thing, it's just that fields is a little more generic. Since blaze handles databases where things are not called columns, such as a mongodb, we decided on using fields for this purpose. 

It also looks like there could be an issue here. I can take a look today. 

Note that passing in fields will not change the underlying data. We could certainly be much mor explicit about this in the docs while simultaneously pointing people to relabel expressions. 

Thanks for the explanation. Actually I was confused about fields being actually a metadata replacement or just convenience aliasing. 

You don't need to use fields/columns.  It's there to provide extra data for data sources that don't name things.  Your dataframe is completely self-describing, so you don't need fields/columns.  You should just do the following

```
d = Data(my_dataframe)
```

The fact that this is silently ignored is an error though.

:+1:  IMO the main issue is really the output from np.asarray() being entirely different.  

Ah, yes, I glossed over that part while reading.  The two issues are related though.  Everything works fine if the field names match up.  I think the right thing to do here is to err when fields is used unnecessarily. 

``` python
In [1]: vs = {'first': [2., 5., 3.], 'second': [4., 1., 4.], 'third': [6., 4., 3.]}

In [2]: import pandas as pd

In [3]: vdf = pd.DataFrame(vs)

In [4]: from blaze import Data

In [5]: d = Data(vdf)

In [6]: d
Out[6]: 
   first  second  third
0      2       4      6
1      5       1      4
2      3       4      3

In [7]: import numpy as np

In [8]: np.asarray(d)
Out[8]: 
array([(2.0, 4.0, 6.0), (5.0, 1.0, 4.0), (3.0, 4.0, 3.0)], 
      dtype=[('first', '<f8'), ('second', '<f8'), ('third', '<f8')])

In [9]: d = Data(vdf, fields=['a', 'b', 'c'])

In [10]: np.asarray(d)
Out[10]: 
array([(0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)], 
      dtype=[('a', '<f8'), ('b', '<f8'), ('c', '<f8')])
```


Resolving patch:
diff --git a/blaze/interactive.py b/blaze/interactive.py
--- a/blaze/interactive.py
+++ b/blaze/interactive.py
@@ -77,6 +77,17 @@ def Data(data, dshape=None, name=None, fields=None, columns=None, schema=None,
             schema = Record(list(zip(fields, types)))
             dshape = DataShape(*(dshape.shape[:-1] + (schema,)))
         elif isrecord(dshape.measure) and fields:
+            ds = discover(data)
+            assert isrecord(ds.measure)
+            names = ds.measure.names
+            if names != fields:
+                raise ValueError('data column names %s\n'
+                                 '\tnot equal to fields parameter %s,\n'
+                                 '\tuse Data(data).relabel(%s) to rename fields'
+                                 % (names,
+                                    fields,
+                                    ', '.join('%s=%r' % (k, v)
+                                              for k, v in zip(names, fields))))
             types = dshape.measure.types
             schema = Record(list(zip(fields, types)))
             dshape = DataShape(*(dshape.shape + (schema,)))


Test patch:
diff --git a/blaze/tests/test_interactive.py b/blaze/tests/test_interactive.py
--- a/blaze/tests/test_interactive.py
+++ b/blaze/tests/test_interactive.py
@@ -359,3 +359,12 @@ def test_highly_nested_repr():
     data = [[0, [[1, 2], [3]], 'abc']]
     d = Data(data)
     assert 'abc' in repr(d.head())
+
+
+def test_asarray_fails_on_different_column_names():
+    vs = {'first': [2., 5., 3.],
+          'second': [4., 1., 4.],
+          'third': [6., 4., 3.]}
+    df = pd.DataFrame(vs)
+    with pytest.raises(ValueError):
+        Data(df, fields=list('abc'))