Repository: pandas-dev/pandas
Pull request: TST: Test for the Enum triggering TypeError (#22551 issue)
- [x] closes #22551



Problem statement:
columns comparison with Enum column name triggers TypeError
#### Code Sample, a copy-pastable example if possible

```python
from enum import Enum
import pandas as pd

Cols = Enum('Cols', 'col1 col2')

q1 = pd.DataFrame({Cols.col1: [1, 2, 3]})
q2 = pd.DataFrame({Cols.col1: [1, 2, 3]})

print((q1[Cols.col1] == q2[Cols.col1]).all())
```
#### Problem description

On pandas 0.23.x, comparison of columns in DataFrames which have enum.Enum column names triggers off a TypeError exception. it looks that pandas calls user defined Enum class as a function. 

```
Traceback (most recent call last):
  File "./pandas_enum.py", line 9, in <module>
    print((q1[Cols.col1] == q2[Cols.col1]).all())
  File "/.../lib/python3.6/site-packages/pandas/core/ops.py", line 1217, in wrapper
    name=res_name).rename(res_name)
  File "/.../lib/python3.6/site-packages/pandas/core/series.py", line 3318, in rename
    return super(Series, self).rename(index=index, **kwargs)
  File "/.../lib/python3.6/site-packages/pandas/core/generic.py", line 973, in rename
    level=level)
  File "/.../lib/python3.6/site-packages/pandas/core/internals.py", line 3340, in rename_axis
    obj.set_axis(axis, _transform_index(self.axes[axis], mapper, level))
  File "/.../lib/python3.6/site-packages/pandas/core/internals.py", line 5298, in _transform_index
    items = [func(x) for x in index]
  File "/.../lib/python3.6/site-packages/pandas/core/internals.py", line 5298, in <listcomp>
    items = [func(x) for x in index]
TypeError: 'Cols' object is not callable
```

#### Expected Output

just print `True`. Pandas 0.22.0 does the expected behavior. Pandas 0.23.0 and later has this problem. 

#### Output of ``pd.show_versions()``

<details>

```
>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.6.1.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-327.4.5.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.4
pytest: None
pip: 18.0
setuptools: 40.2.0
Cython: None
numpy: 1.15.1
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
```

</details>

Hints from issue discussion:
Hmm is there any reason why you wouldn't just use the `.name` of your Enum? Would seem like a much safer approach not sure we want to (or rather can) support indexing by arbitrary objects
My motivation is the same with using `Symbol()` objects as property keys in Javascript (ES6). It can be an explicit key for both coding and debugging and can also avoid name conflict.
I have the same problem, and would also prefer not to use the .name-property to ensure specificity of the key.

Or rather: If the reason for this error is an unintended effect of a change in 0.22 to 0.23, what would speak against restoring the original behavior?
Same issue: https://stackoverflow.com/q/55822555/1240268
I get a similar issue but I am not indexing at all the dataframe or the series, just comparing it

```
from uuid import uuid4

import pandas as pd

col = uuid4()

sr = pd.Series([4, 5, 6], name=col)
(sr == sr)
# raise TypeError: 'UUID' object is not callable

df = pd.DataFrame({col: [1, 2, 3]})
(df == df)
# raise TypeError: 'UUID' object is not callable
```
Looks to work on master. Could use a test

```
In [23]: from enum import Enum
    ...: import pandas as pd
    ...:
    ...: Cols = Enum('Cols', 'col1 col2')
    ...:
    ...: q1 = pd.DataFrame({Cols.col1: [1, 2, 3]})
    ...: q2 = pd.DataFrame({Cols.col1: [1, 2, 3]})
    ...:
    ...: print((q1[Cols.col1] == q2[Cols.col1]).all())
True
```
Hey! I am a first-time contributor and would like to help on this. My experience with tests is pretty non-existant, so as far as I understand it one should convert the code-example to a function, and then add this to the appropriate pandas/test suite (maybe to pandas/test/dtypes)?

Commit message: updated formatting

Patch to review:
diff --git a/pandas/tests/frame/indexing/test_get_value.py b/pandas/tests/frame/indexing/test_get_value.py
index 65a1c64a1578a..c88977e0ffe24 100644
--- a/pandas/tests/frame/indexing/test_get_value.py
+++ b/pandas/tests/frame/indexing/test_get_value.py
@@ -1,3 +1,5 @@
+from enum import Enum
+
 import pytest
 
 from pandas import (
@@ -20,3 +22,16 @@ def test_get_value(self, float_frame):
                 result = float_frame._get_value(idx, col)
                 expected = float_frame[col][idx]
                 assert result == expected
+
+
+def test_enum_value():
+    Cols = Enum("Cols", "col1 col2")
+
+    q1 = DataFrame({Cols.col1: [1, 2, 3]})
+    q2 = DataFrame({Cols.col1: [1, 2, 3]})
+
+    result = (q1[Cols.col1] == q2[Cols.col1]).all()
+
+    expected = True
+
+    assert result == expected


Code review comments:
diff_hunk: @@ -20,3 +22,16 @@ def test_get_value(self, float_frame):
                 result = float_frame._get_value(idx, col)
                 expected = float_frame[col][idx]
                 assert result == expected
+
+
+def test_enum_value():
path: pandas/tests/frame/indexing/test_get_value.py
text: @user1:
1. Could you name this test `test_enum_column_equality`
2. Could you move this to `pandas/tests/frame/test_arithmetic.py`?
3. Also below could you structure the test like
```
result = q1[Cols.col1] == q2[Cols.col1]
expected = ...
tm.assert_equal(result, expected)
```


@author:
Sure! Thank you for your guide.

@author:
@user1 by `result = q1[Cols.col1] == q2[Cols.col1]` do you mean that I remove the `.all()` method as well?

@user1:
Correct

@user1:
The changes in this file can be removed now.

@author:
@user1
Sure. About removing `a.all()` when I create a Pandas.Series with the same truth values to compare with the _result_, it tells me that the truth value of a series is ambiguous and I should use either of these `a.empty, a.bool(), a.item(), a.any() or a.all().`

@user1:
If you inspect `result`, it's a Series

```
In [1]:  from enum import Enum

In [2]:     Cols = Enum("Cols", "col1 col2")
   ...:
   ...:     q1 = DataFrame({Cols.col1: [1, 2, 3]})
   ...:     q2 = DataFrame({Cols.col1: [1, 2, 3]})
   ...:
   ...:     result = q1[Cols.col1] == q2[Cols.col1]

In [3]: result
Out[3]:
0    True
1    True
2    True
Name: Cols.col1, dtype: bool
```

So `expected` should also be a `Series`

@author:
@user1 
Right. but when I created the series and tried to compare, 

```
data = np.array([True, True, True])
expected = pd.Series(data)
```
checking if `expected == result` gave me the Error that the truth value of the Series is ambiguous, that is why. Unless `tm.assert_equal` accounts for that. update: it did not.

@user1:
Yup, `tm.assert_equal` will check the values and the aspects of the Series

```
In [1]:  from enum import Enum

In [2]:     Cols = Enum("Cols", "col1 col2")
   ...:
   ...:     q1 = DataFrame({Cols.col1: [1, 2, 3]})
   ...:     q2 = DataFrame({Cols.col1: [1, 2, 3]})
   ...:
   ...:     result = q1[Cols.col1] == q2[Cols.col1]

In [3]: result
Out[3]:
0    True
1    True
2    True
Name: Cols.col1, dtype: bool

In [4]: import pandas._testing as tm

In [5]: epxected = pd.Series([True, True, True], name=Cols.col1)

In [6]: tm.assert_series_equal(result, epxected)
```

@author:
Update: changing it to `tm.assert_series_equal(result, expected)` , hoping it works.

@author:
Oh, I've just seen your reply. There was a syntax issue that I fixed. `pd.Series` now replaced with `Series` which hopefully should be working now. Thank you for your guidance as well.