Add Zstandard compression support
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

Haven't found any discussion about Zstandard so far but I think it would be a valuable addition to the Pickle compression options.

Labels:
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
Hello @jonashaag! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

































































There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-12-22 09:44:38 UTC
Thanks for the review!

- Added Zstandard also to CSV writing and other places and updated all the docstrings
- Refactored some tests to use `_compression_to_extension` dictionary
- Fixed binary mode detection in `io.common` for Zstandard.
cc @twoertwein if you can have a look
Will push an update soon 
I've consolidated all `compression` docstrings into two shared fragments
If this builds OK it should be ready for re-review
Test failures should go away when this PR is merged (missing `.zst` URL in this GitHub repo)
You will need to add "zstd" here https://github.com/pandas-dev/pandas/blob/85c221a050d548fcb0541f3153372b8ca3aee70e/pandas/_typing.py#L246 to make mypy happy.
The only obvious part missing to me is a whatsnew entry.
@jreback ping, should be good to go now. Test failures are because one of the test only works when this has been merged to `master`.
Something with CI is really broken, all the PRs are red
@jreback should be good to go 
can you merge master once more, been having a number of CI issues
https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=71120&view=logs&j=9dd5e044-6afa-5be6-cb56-860f1ee82b31&t=d3ae057b-d0af-59ce-6e4c-b69ef3739701 are failing related to zstd
@jreback looks like those are only failing because those files are not yet on `master`
>  failing because those files are not yet on master

what files?
> what files?

```
2021-12-22T10:21:51.1829969Z _______________ test_compressed_urls[python-explicit-zstd-.zst] _______________
...
2021-12-22T10:21:51.1885656Z E       urllib.error.HTTPError: HTTP Error 404: Not Found
2021-12-22T10:21:51.1885903Z 
2021-12-22T10:21:51.1886527Z C:\Miniconda\envs\pandas-dev\lib\urllib\request.py:649: HTTPError
2021-12-22T10:21:51.1887052Z ________________ test_compressed_urls[python-infer-zstd-.zst] _________________
```

https://github.com/pandas-dev/pandas/tree/master/pandas/tests/io/parser/data

https://github.com/pandas-dev/pandas/blob/5f36af325c276454201495eaf66be55bd17964a5/pandas/tests/io/parser/test_network.py#L38-L43

Hardcoded URL access to master branch 
thanks @jonashaag very nice!
Looks like master is failing some builds are missing zstandard (or not `importskip`ed)?

https://github.com/pandas-dev/pandas/runs/4612486852?check_suite_focus=true#step:10:255
Yep
```
FAILED pandas/tests/io/parser/test_network.py::test_compressed_urls[python-explicit-zstd-.zst]
FAILED pandas/tests/io/parser/test_network.py::test_compressed_urls[python-infer-zstd-.zst]
FAILED pandas/tests/io/parser/test_network.py::test_compressed_urls[c-explicit-zstd-.zst]
FAILED pandas/tests/io/parser/test_network.py::test_compressed_urls[c-infer-zstd-.zst]
```