Repository: dask/dask

Issue:
to_parquet partitioned by 'date'
I'm trying to save a dask dataframe `to_parquet` and partition it by date. The system won't allow the naming of the file `date=2022-01-01 00:00:00`. 
```
dates = pd.DataFrame({'date':pd.date_range(start='2022-01-01', end='2022-01-31', freq='D')})
ddata = dd.from_pandas(dates, npartitions=1)
ddata.to_parquet("dates.pq", partition_on="date")
```
```
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '***/dask/dates.pq/date=2022-01-01 00:00:00'
```

Getting rid of the ` 00:00:00` would solve the issue. 

**Workaround**:
Save `date` as a string, but I would like it to be a date. 

I'm using dask '2023.5.1'


Discussion:
Thanks for the report @marcdelabarrera!  I cannot reproduce on my end, so I'm thinking this is probably specific to windows. Can you confirm if you are using the "pyarrow" engine (by specifying `engine="pyarrow"`)?
Thank you for the reply. @rjzamora. 

I tried with `pyarrow` and I am getting the same error. I tried the same on Linxu and indeed worked. The problem is that on Windows, `:` is an invalid character to write in a directory name, while it does not seem to be the case with other OS.

I guess a partial fix is that the file was saved as `yyyy-mm-dd`, although this would drop information regarding hours, minutes, and seconds, or replacing `:` for some other allowed character.
>The problem is that on Windows, : is an invalid character to write in a directory name

Yes, you are definitely correct about this. I'll try to see if pyarrow's native `write_to_dataset` (or other systems) do something special to handle this problem.
>I guess a partial fix is that the file was saved as yyyy-mm-dd, although this would drop information regarding hours, minutes, and seconds, or replacing : for some other allowed character.

I'm not really finding any convenient ways to automatically handle the `":"` problem while partitioning by a `date_range` column. In practice, most hive partitioning tends to be on a specific "day", "month" or "year" (the second is not typically something you want to partition on). Therefore, if you need to track the time information somewhere, you may want to split your `"date"` column into distinct `"date"` and `"time"` columns, and only use the `"date"` column for partitioning.  If you don't need the time information, then doing the following before the write is probably your best bet:

```python
dates["date"] = dates["date"].astype("date64[pyarrow]")
```

Patch:
diff --git a/dask/dataframe/io/parquet/core.py b/dask/dataframe/io/parquet/core.py
--- a/dask/dataframe/io/parquet/core.py
+++ b/dask/dataframe/io/parquet/core.py
@@ -317,10 +317,8 @@ def read_parquet(
         set to 'infer' or 'adaptive'. Default may be engine-dependant, but is
         128 MiB for the 'pyarrow' and 'fastparquet' engines.
     aggregate_files : bool or str, default None
-        WARNING: The ``aggregate_files`` argument will be deprecated in the future.
-        Please consider using ``from_map`` to create a DataFrame collection with a
-        custom file-to-partition mapping. If you strongly oppose the deprecation of
-        ``aggregate_files``, comment at https://github.com/dask/dask/issues/9051".
+        WARNING: Passing a string argument to ``aggregate_files`` will result
+        in experimental behavior. This behavior may change in the future.
 
         Whether distinct file paths may be aggregated into the same output
         partition. This parameter is only used when `split_row_groups` is set to
@@ -416,14 +414,11 @@ def read_parquet(
             FutureWarning,
         )
 
-    # "Pre-deprecation" warning for `aggregate_files`
-    if aggregate_files:
+    # FutureWarning for `aggregate_files`
+    if aggregate_files and isinstance(aggregate_files, str):
         warnings.warn(
-            "The `aggregate_files` argument will be deprecated in the future. "
-            "Please consider using `from_map` to create a DataFrame collection "
-            "with a custom file-to-partition mapping.\n\n"
-            "If you strongly oppose the deprecation of `aggregate_files`, "
-            "please comment at https://github.com/dask/dask/issues/9051",
+            "String support for `aggregate_files` is experimental. "
+            "Behavior may change in the future. ",
             FutureWarning,
         )
 


Test patch:
diff --git a/dask/dataframe/io/tests/test_parquet.py b/dask/dataframe/io/tests/test_parquet.py
--- a/dask/dataframe/io/tests/test_parquet.py
+++ b/dask/dataframe/io/tests/test_parquet.py
@@ -2861,13 +2861,12 @@ def test_split_row_groups_int_aggregate_files(tmpdir, engine, split_row_groups):
 
     # Read back with both `split_row_groups>1` and
     # `aggregate_files=True`
-    with pytest.warns(FutureWarning, match="argument will be deprecated"):
-        ddf2 = dd.read_parquet(
-            str(tmpdir),
-            engine=engine,
-            split_row_groups=split_row_groups,
-            aggregate_files=True,
-        )
+    ddf2 = dd.read_parquet(
+        str(tmpdir),
+        engine=engine,
+        split_row_groups=split_row_groups,
+        aggregate_files=True,
+    )
 
     # Check that we are aggregating files as expected
     npartitions_expected = math.ceil((size / row_group_size) / split_row_groups)
@@ -3052,13 +3051,23 @@ def test_split_adaptive_files(
         write_index=False,
     )
 
-    with pytest.warns(FutureWarning, match="argument will be deprecated"):
+    aggregate_files = partition_on if partition_on else True
+    if isinstance(aggregate_files, str):
+        with pytest.warns(FutureWarning, match="Behavior may change"):
+            ddf2 = dd.read_parquet(
+                str(tmpdir),
+                engine=read_engine,
+                blocksize=blocksize,
+                split_row_groups="adaptive",
+                aggregate_files=aggregate_files,
+            )
+    else:
         ddf2 = dd.read_parquet(
             str(tmpdir),
             engine=read_engine,
             blocksize=blocksize,
             split_row_groups="adaptive",
-            aggregate_files=partition_on if partition_on else True,
+            aggregate_files=aggregate_files,
         )
 
     # Check that files where aggregated as expected
@@ -3103,7 +3112,7 @@ def test_split_adaptive_aggregate_files(
         partition_on=partition_on,
         write_index=False,
     )
-    with pytest.warns(FutureWarning, match="argument will be deprecated"):
+    with pytest.warns(FutureWarning, match="Behavior may change"):
         ddf2 = dd.read_parquet(
             str(tmpdir),
             engine=read_engine,
@@ -3157,16 +3166,15 @@ def test_split_adaptive_blocksize(tmpdir, blocksize, engine, metadata):
         assert "_metadata" not in files
         path = os.path.join(dirname, "*.parquet")
 
-    with pytest.warns(FutureWarning, match="argument will be deprecated"):
-        ddf2 = dd.read_parquet(
-            path,
-            engine=engine,
-            blocksize=blocksize,
-            split_row_groups="adaptive",
-            calculate_divisions=True,
-            index="index",
-            aggregate_files=True,
-        )
+    ddf2 = dd.read_parquet(
+        path,
+        engine=engine,
+        blocksize=blocksize,
+        split_row_groups="adaptive",
+        calculate_divisions=True,
+        index="index",
+        aggregate_files=True,
+    )
 
     assert_eq(ddf1, ddf2, check_divisions=False)