Repository: bokeh/bokeh

Issue:
TimeSeries modifies a pd.DataFrame inplace
From the ML - https://groups.google.com/a/continuum.io/forum/#!topic/bokeh/yF77ulMnE6Q

``` python
df1 = pd.DataFrame(data=np.random.rand(5,3), columns = ('a', 'b' ,'c'), index = pd.DatetimeIndex(start='01-01-2015',periods=5, freq='d'))   
df2 = df1.copy()
assert df2.equals(df1)
TimeSeries(df2)
assert df2.equals(df1)
```

results in:

``` python-traceback
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-32-8de7d5ef3815> in <module>()
      3 assert df2.equals(df1)
      4 TimeSeries(df2)
----> 5 assert df2.equals(df1)

AssertionError: 
```

The reason being that the call to TimeSeries has appended an index column:

```
In [36]: df2.columns 
Out [36]:  Index(['a', 'b', 'c', 'index'], dtype='object')
```



Discussion:
I think it's a good idea for an API to not change the object passed unless it is a stated side-effect.  This is a question better answered by the Bokeh developers.

The client side workaround is to pass a copy of the DataFrame.

```
TimeSeries(df2.copy())
```

But that's not a good long term solution IMO.

The library side fix is likely to make a copy of the DataFrame when storing the data object.  This might introduce memory issues when passing a very large DataFrame, but I don't know how much of a concern this is.  Also, I don't know if it is actually desirable for the chart object to keep the reference to the object rather than a copy.  I would defer to the core developers who know this much better than myself.

The code that is adding the column is in `setup_derived_columns`
https://github.com/bokeh/bokeh/blob/0.11.1/bokeh/charts/data_source.py#L410

BTW, I found this by taking the example above and using a DataFrame subclass that would raise on adding a new column.

```
import pandas as pd
import numpy as np
from bokeh.charts import TimeSeries
df1 = pd.DataFrame(data=np.random.rand(5,3), columns = ('a', 'b' ,'c'), index = pd.DatetimeIndex(start='01-01-2015',periods=5, freq='d'))   

class DataFrameRaiseOnSetItem(pd.DataFrame):
    def __setitem__(self, key, value):
        raise RuntimeError('doh!')

df = DataFrameRaiseOnSetItem(df1)
TimeSeries(df)
```

which raised this exception

```
RuntimeError                              Traceback (most recent call last)
<ipython-input-14-ed0ca717fc89> in <module>()
----> 1 TimeSeries(df)

/usr/local/lib/python3.4/dist-packages/bokeh/charts/builders/timeseries_builder.py in TimeSeries(data, x, y, builder_type, **kws)
    100     kws['x'] = x
    101     kws['y'] = y
--> 102     return create_and_build(builder_type, data, **kws)

/usr/local/lib/python3.4/dist-packages/bokeh/charts/builder.py in create_and_build(builder_class, *data, **kws)
     60     # create the new builder
     61     builder_kws = {k: v for k, v in kws.items() if k in builder_props}
---> 62     builder = builder_class(*data, **builder_kws)
     63 
     64     # create a chart to return, since there isn't one already

/usr/local/lib/python3.4/dist-packages/bokeh/charts/builder.py in __init__(self, *args, **kws)
    273             data_args['attrs'] = attrs
    274             data_args['column_assigner'] = self.column_selector
--> 275             data = ChartDataSource.from_data(*args, **data_args)
    276 
    277             # make sure that the builder dimensions have access to the chart data source

/usr/local/lib/python3.4/dist-packages/bokeh/charts/data_source.py in from_data(cls, *args, **kwargs)
    693             elif isinstance(table, pd.DataFrame):
    694                 kwargs['input_type'] = 'DataFrame'
--> 695                 return cls(df=table, **kwargs)
    696 
    697             # unrecognized input type

/usr/local/lib/python3.4/dist-packages/bokeh/charts/data_source.py in __init__(self, df, dims, required_dims, selections, column_assigner, attrs, **kwargs)
    327                                                attrs=self.attrs)
    328         self._selections = self.get_selections(selections, **kwargs)
--> 329         self.setup_derived_columns()
    330         self.apply_operations()
    331         self.meta = self.collect_metadata(df)

/usr/local/lib/python3.4/dist-packages/bokeh/charts/data_source.py in setup_derived_columns(self)
    409                     self.df.columns.tolist():
    410                         self._data[dim_selection] = special_columns[dim_selection](
--> 411                             self._data)
    412 
    413     def __getitem__(self, dim):

<ipython-input-13-7bf6cb9e25aa> in __setitem__(self, key, value)
      1 class DataFrameRaiseOnSetItem(pd.DataFrame):
      2     def __setitem__(self, key, value):
----> 3         raise RuntimeError('doh!')
      4 
      5 df = DataFrameRaiseOnSetItem(df1)

RuntimeError: doh!
```

I agree it's not optimal to do without any warning, but also agree for large data frames it can potentially be wasteful. Perhaps there is a way to avoid it altogether (not sure) or if not, we could add an "inplace" or similar parameter to the chart that controls whether it is allowed to add the index, or if if must make a copy. 

A [shallow copy](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.copy.html#pandas-dataframe-copy) would probably do the job.

``` python
In [8]: df1 = pd.DataFrame(randn(5,3))
   ...: df1
Out[8]: 
        0       1       2
0 -0.1447  0.2795  0.2067
1  0.3035 -0.5123 -2.1199
2 -1.7369  0.2083  0.9238
3 -0.3963  1.1770  2.3644
4  0.0099  1.0266  0.1318

In [9]: df2 = df1.copy(deep=False)
   ...: df2['index'] = np.arange(5)[::-1]
   ...: df2
Out[9]: 
        0       1       2  index
0 -0.1447  0.2795  0.2067      4
1  0.3035 -0.5123 -2.1199      3
2 -1.7369  0.2083  0.9238      2
3 -0.3963  1.1770  2.3644      1
4  0.0099  1.0266  0.1318      0

In [10]: df1
Out[10]: 
        0       1       2
0 -0.1447  0.2795  0.2067
1  0.3035 -0.5123 -2.1199
2 -1.7369  0.2083  0.9238
3 -0.3963  1.1770  2.3644
4  0.0099  1.0266  0.1318

In [11]: df2.iloc[0,0] = 99
    ...: df2
Out[11]: 
         0       1       2  index
0  99.0000  0.2795  0.2067      4
1   0.3035 -0.5123 -2.1199      3
2  -1.7369  0.2083  0.9238      2
3  -0.3963  1.1770  2.3644      1
4   0.0099  1.0266  0.1318      0

In [12]: df1
Out[12]: 
         0       1       2
0  99.0000  0.2795  0.2067
1   0.3035 -0.5123 -2.1199
2  -1.7369  0.2083  0.9238
3  -0.3963  1.1770  2.3644
4   0.0099  1.0266  0.1318
```

I had no idea that existed, this is why more contributors are awesome. :) @dennisobrien or @dhirschfeld would one of you care to work up a PR to change use the shallow copy internally?


Resolving patch:
diff --git a/bokeh/charts/data_source.py b/bokeh/charts/data_source.py
--- a/bokeh/charts/data_source.py
+++ b/bokeh/charts/data_source.py
@@ -151,7 +151,7 @@ def __init__(self, label, data, attr_specs):
             data (:class:`pandas.DataFrame`): the subset of data associated with the group
             attr_specs dict(str, :class:`AttrSpec`): mapping between attribute name and
             the associated :class:`AttrSpec`.
-            
+
         """
         self.label = label
         self.data = data.reset_index()
@@ -319,12 +319,15 @@ def __init__(self, df, dims=None, required_dims=None, selections=None,
 
         self.input_type = kwargs.pop('input_type', None)
         self.attrs = attrs or []
-        self._data = df
+        self._data = df.copy(deep=False)
         self._dims = dims
         self.operations = []
         self._required_dims = required_dims
-        self.column_assigner = column_assigner(df=df, dims=list(self._dims),
-                                               attrs=self.attrs)
+        self.column_assigner = column_assigner(
+            df=self._data,
+            dims=list(self._dims),
+            attrs=self.attrs,
+        )
         self._selections = self.get_selections(selections, **kwargs)
         self.setup_derived_columns()
         self.apply_operations()


Test patch:
diff --git a/bokeh/charts/tests/test_data_source.py b/bokeh/charts/tests/test_data_source.py
--- a/bokeh/charts/tests/test_data_source.py
+++ b/bokeh/charts/tests/test_data_source.py
@@ -41,14 +41,19 @@ def test_array(test_data):
 
 def test_pandas(test_data):
     """Test creating chart data source from existing dataframe."""
-    ds = ChartDataSource.from_data(test_data.pd_data)
+    pd_data = test_data.pd_data.copy()
+    ds = ChartDataSource.from_data(pd_data)
     assert len(ds.columns) == 2
     assert len(ds.index) == 4
 
-    ds = ChartDataSource(test_data.pd_data)
+    ds = ChartDataSource(pd_data)
     assert len(ds.columns) == 2
     assert len(ds.index) == 4
 
+    # Test for issue #3899
+    # ChartDataSource shouldn't modify the input DataFrame
+    assert pd_data.equals(test_data.pd_data)
+
 
 def test_dict(test_data):
     """Test creating chart data source from dict of arrays."""