Repository: dask/dask

Issue:
`to_parquet` compression parameter seems to have no effect
```python
from dask.distributed import Client
import dask.dataframe as dd
client = Client(n_workers=1)

df = dd.read_parquet('path-to-file.parquet')
df.to_parquet('output.parquet', engine='pyarrow', compression='brotli', parition_on=[col])
```

**What happened**:

Choosing `gzip`, `snappy`, `brotli` or `lz4` compression options had no effect on the size of the created parquet file. In fact, the file created with the `to_parquet` method actually _inflated_ the size by a factor of 3 (from ~7GB to ~20GB). This leads me to suspect that, possibly, no compression is being used somehow? No errors or warnings get raised -- **even if `compression` is set to a nonsensical string**. 

I've tried using both `fastparquet` and `pyarrow` as the engine during writing.

Is there something obvious I'm missing?

**What you expected to happen**:
Expected the various compression settings to have an effect on the size of the output parquet file.

**Anything else we need to know?**:
The various compression settings _do_ appear to have an effect when using `pandas.DataFrame.to_parquet`, as well as when used with the `pyarrow.parquet` library. 

**Environment**:

- Dask version: 2.22.0
- Python version: 3.7.3
- Arrow version: 1.0.0
- Operating System: Manjaro Linux
- Install method (conda, pip, source): pip



Discussion:
Thanks for opening this issue! It looks to me like when you use `partition_on` the `compression` option doesn't make it all the way through to the pyarrow command. Specifically, `compression` is missing here:

https://github.com/dask/dask/blob/14c53510a5a58f8eb452c148a97f7813234193d1/dask/dataframe/io/parquet/arrow.py#L78:L78

Here is an example that doesn't throw an error and should:

```python
from dask.datasets import timeseries

ddf = timeseries(freq="1T")
ddf.to_parquet("output.parquet", engine="pyarrow", partition_on=["name"], compression="foo")
```

Resolving patch:
diff --git a/dask/dataframe/io/parquet/arrow.py b/dask/dataframe/io/parquet/arrow.py
--- a/dask/dataframe/io/parquet/arrow.py
+++ b/dask/dataframe/io/parquet/arrow.py
@@ -932,7 +932,14 @@ def write_partition(
         t = pa.Table.from_pandas(df, preserve_index=preserve_index, schema=schema)
         if partition_on:
             md_list = _write_partitioned(
-                t, path, filename, partition_on, fs, index_cols=index_cols, **kwargs
+                t,
+                path,
+                filename,
+                partition_on,
+                fs,
+                index_cols=index_cols,
+                compression=compression,
+                **kwargs,
             )
             if md_list:
                 _meta = md_list[0]


Test patch:
diff --git a/dask/dataframe/io/tests/test_parquet.py b/dask/dataframe/io/tests/test_parquet.py
--- a/dask/dataframe/io/tests/test_parquet.py
+++ b/dask/dataframe/io/tests/test_parquet.py
@@ -1455,6 +1455,20 @@ def test_writing_parquet_with_compression(tmpdir, compression, engine):
     check_compression(engine, fn, compression)
 
 
+@pytest.mark.parametrize("compression,", ["default", None, "gzip", "snappy"])
+def test_writing_parquet_with_partition_on_and_compression(tmpdir, compression, engine):
+    fn = str(tmpdir)
+    if compression in ["snappy", "default"]:
+        pytest.importorskip("snappy")
+
+    df = pd.DataFrame({"x": ["a", "b", "c"] * 10, "y": [1, 2, 3] * 10})
+    df.index.name = "index"
+    ddf = dd.from_pandas(df, npartitions=3)
+
+    ddf.to_parquet(fn, compression=compression, engine=engine, partition_on=["x"])
+    check_compression(engine, fn, compression)
+
+
 @pytest.fixture(
     params=[
         # fastparquet 0.1.3