ENH: .read_pickle(...) from zip containing hidden OS X/macOS metadata files/folders
State: closed

- [x] closes #37098 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This PR allows for `.zip` files created by OS X and macOS that contain `__MACOSX` and `.DS_STORE` metadata folders to be loaded by `read_pickle/read_csv/read_table/read_json` without error. 

~It does **not** work with `pd.read_csv(...)` as in that case the compression is handled in the C code. If this enhancement is deemed desirable then I'm willing to have a go at writing it (and the test is already written).~

(No longer true as of https://github.com/pandas-dev/pandas/pull/36997)

Other similar folders may exist from other operating systems, in which case the list could be pulled out as a constant which could be used in the tests and in the module itself.

Labels:

Comments:
Hello @ml-evs! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-11-19 15:01:58 UTC
I cannot comment on whether this PR is 'desirable' :)

I think it it should also  work for `pd.read_csv(..., engine="python")`, or? And if I manage to get https://github.com/pandas-dev/pandas/pull/36997 working, it will also work with the c engine.
> I cannot comment on whether this PR is 'desirable' :)

I wouldn't want to argue particularly hard for it myself! I guess it depends how prevalent this "problem" is, but I won't be too upset if this doesn't make it in :upside_down_face: 

> I think it it should also work for `pd.read_csv(..., engine="python")`, or? And if I manage to get #36997 working, it will also work with the c engine.

It does indeed work with the Python engine, the tests in `tests/io/parsers/test_compression.py` are run on every engine, I'm happy to push the failing tests if that would useful. Can also rebase onto your PR and check if that works too. 

I'll hold off until there's been a discussion about whether this is desirable (either here or in the issue #37098).


> It does indeed work with the Python engine, the tests in `tests/io/parsers/test_compression.py` are run on every engine, I'm happy to push the failing tests if that would useful. Can also rebase onto your PR and check if that works too.
> 
> I'll hold off until there's been a discussion about whether this is desirable (either here or in the issue #37098).

I wouldn't wait for my PR (I'm not sure how long it will take me to debug two failing tests on windows). I will ping you when/if the PR gets merged. Including test for the python engine would be good.

> Is this only an issue on read? Not write?

I think it's only a problem when the zip has been made inside the macOS file manager, rather than say, at the command-line (though I do not have a Mac to test this).

As I said before, maybe this is a non-issue.
@ml-evs #36997 is merged: you PR will also work for `read_csv` (both c engine and python engine).
> @ml-evs #36997 is merged: you PR will also work for `read_csv` (both c engine and python engine).

This seems to be the case, thanks @twoertwein!
Happy to close this if there's no interest; will re-ping the initial reviewers just in case.
ive tried, but have been unable to form an opinion
a more generic approach might be to use `{'method': 'zip', 'file': 'path/within/zip'}` to let the user choose which file in a ZIP file should be used? This is more work for the user but allows to read different files within a ZIP file.

Technically this can already be achieved with 
```py
from zipfile import ZipFile

import pandas as pd

with ZipFile("test.zip", mode="r") as file:
    with file.open("data.pickle", mode="r") as file:
        pd.read_pickle(file)
```