Repository: pandas-dev/pandas
Pull request: TST: GH20676 Verify equals operator for list of Numpy arrays
- [x] closes #20676
- [ x ] tests added / passed
- [ x ] passes `black pandas`
- [ x ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ x ] whatsnew entry


Problem statement:
pd.Series.equals() fails for Series containing iterable objects
#### Code Sample, a copy-pastable example if possible

```python
pd.Series(
    [np.array([1, 2]), np.array([1, 2])]
).equals(pd.Series(
    [np.array([1, 2]), np.array([1, 2])]
))

# Evaluates to False
```
#### Problem description

For objects that overload equality with elementwise equality (this includes `np.array` and `pd.Series` objects), `pd.Series.equals()` catches

```python
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

and returns `False`, behavior inherited from `np.equal` applied to object arrays. While you can argue this is a numpy bug, in numpy equality is purposefully strict for `dtype=='object'`. In pandas, it is a common enough procedure to tabulate arrays and lists under a single column (which may contain a named (time)series, image array, or an unpacked NLP document), and this behavior of equality is unintuitive.

#### Expected Output
`True`
#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.13.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-1048-aws
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.22.0
pytest: 3.2.2
pip: 9.0.1
setuptools: 27.2.0
Cython: None
numpy: 1.13.3
scipy: 0.19.1
pyarrow: None
xarray: None
IPython: 5.3.0
sphinx: 1.5.4
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.1.9
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

</details>

Hints from issue discussion:
Storing lists or arrays in Series/DataFrame is generally not fully supported (in the sense that: it works for basic things, but may fail for corner cases (eg indexing, setting values, ..) and it's not well tested, and it's not something the core devs will typically give priority to).

But if you have an relative straightforward fix, we will be happy to take a PR.

Note that numpy also does not handle equality well for object arrays of arrays:

```
In [62]: s = pd.Series([np.array([1, 2]), np.array([1, 2])])

In [63]: s.values
Out[63]: array([array([1, 2]), array([1, 2])], dtype=object)

In [64]: s.values == s.values
/home/joris/miniconda3/envs/dev/bin/ipython:1: DeprecationWarning: elementwise == comparison failed; this will raise an error in the future.
  #!/home/joris/miniconda3/envs/dev/bin/python
Out[64]: True
```
Just a thought: it might be that exploring an ExtensionArray (new in master: http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#extending-pandas-with-custom-types) might be a way to fix this.
yeah this is really a numpy issue as that is how we compare object arrays. If we need to peer inside those arrays we have to do an inference check, which could be done here. Storing list-likes inside a pandas cell is not supported at all and causes lots of pain in our codebase (think indexing). Using an extension array would be a real nice usecase here.
This looks to work on master. I guess it could use a test

```
In [50]: pd.Series(
    ...:     [np.array([1, 2]), np.array([1, 2])]
    ...: ).equals(pd.Series(
    ...:     [np.array([1, 2]), np.array([1, 2])]
    ...: ))
Out[50]: True

In [51]: pd.__version__
Out[51]: '1.1.0.dev0+1216.gd4d58f960'
```
take

Commit message: TST: GH20676 Verify equals operator for list of Numpy arrays

Patch to review:
diff --git a/pandas/tests/series/methods/test_equals.py b/pandas/tests/series/methods/test_equals.py
new file mode 100644
index 0000000000000..cda888fd39306
--- /dev/null
+++ b/pandas/tests/series/methods/test_equals.py
@@ -0,0 +1,13 @@
+import numpy as np
+import pytest
+
+from pandas import Series
+
+
+@pytest.mark.parametrize(
+    "input_data", [[np.array([1, 2]), np.array([1, 2])]],
+)
+def test_equals_list_array(input_data):
+    # GH20676 Verify equals operator for list of Numpy arrays
+    ser = Series(input_data)
+    assert ser.equals(ser)


Code review comments:
diff_hunk: @@ -0,0 +1,13 @@
+import numpy as np
+import pytest
+
+from pandas import Series
+
+
+@pytest.mark.parametrize(
+    "input_data", [[np.array([1, 2]), np.array([1, 2])]],
path: pandas/tests/series/methods/test_equals.py
text: @user1:
You can in-line this data in the test since is parameterizing over 1 value