Support categorical data types for Parquet
Labels: name: enhancement
description: New feature or request

### Feature request

Huggingface datasets does not seem to support categorical / dictionary data types for Parquet as of now. There seems to be a `TODO` in the code for this feature but no implementation yet. Below you can find sample code to reproduce the error that is currently thrown when attempting to read a Parquet file with categorical columns:

```python
import pandas as pd
import pyarrow.parquet as pq

from datasets import load_dataset

# Create categorical sample DataFrame
df = pd.DataFrame({'type': ['foo', 'bar']}).astype('category')
df.to_parquet('data.parquet')

# Read back as pyarrow table
table = pq.read_table('data.parquet')
print(table.schema)
# type: dictionary<values=string, indices=int32, ordered=0>

# Load with huggingface datasets
load_dataset('parquet', data_files='data.parquet')
```

Error:
```
Traceback (most recent call last):                                                                                                                                                                                                        
  File ".venv/lib/python3.10/site-packages/datasets/builder.py", line 1875, in _prepare_split_single
    writer.write_table(table)
  File ".venv/lib/python3.10/site-packages/datasets/arrow_writer.py", line 566, in write_table
    self._build_writer(inferred_schema=pa_table.schema)
  File ".venv/lib/python3.10/site-packages/datasets/arrow_writer.py", line 379, in _build_writer
    inferred_features = Features.from_arrow_schema(inferred_schema)                                                                                     
  File ".venv/lib/python3.10/site-packages/datasets/features/features.py", line 1622, in from_arrow_schema
    obj = {field.name: generate_from_arrow_type(field.type) for field in pa_schema}
  File ".venv/lib/python3.10/site-packages/datasets/features/features.py", line 1622, in <dictcomp>
    obj = {field.name: generate_from_arrow_type(field.type) for field in pa_schema}
  File ".venv/lib/python3.10/site-packages/datasets/features/features.py", line 1361, in generate_from_arrow_type
    raise NotImplementedError  # TODO(thom) this will need access to the dictionary as well (for labels). I.e. to the py_table
NotImplementedError
```

### Motivation

Categorical data types, as offered by Pandas and implemented with the `DictionaryType` dtype in `pyarrow` can significantly reduce dataset size and are a handy way to turn textual features into numerical representations and back. Lack of support in Huggingface datasets greatly reduces compatibility with a common Pandas / Parquet feature.

### Your contribution

I could provide a PR. However, it would be nice to have an initial complexity estimate from one of the core developers first.

Comments:
Hi ! We could definitely a type that holds the categories and uses a DictionaryType storage. There's a ClassLabel type that is similar with a 'names' parameter (similar to a id2label in deep learning frameworks) that uses an integer array as storage.

It can be added in `features.py`. Here are some pointers:
- the conversion from HF type to PyArrow type is done in `get_nested_type`
- the conversion from Pyarrow type to HF type is done in `generate_from_arrow_type`
- `encode_nested_example` and `decode_nested_example` are used to do user's value (what users see) <-> storage value (what is in the pyarrow.array) if there's any conversion to do

@kklemon did you implement this? Otherwise I would like to give it a try

@mhattingpete no, I hadn't time for this so far. Feel free to work on this.

#self-assign