GitHub issue #3686: `Translation` features cannot be `flatten`ed
State: closed | Created: 2022-02-08T11:33:48Z

## Describe the bug

(`Dataset.flatten`)[https://github.com/huggingface/datasets/blob/master/src/datasets/arrow_dataset.py#L1265] fails for columns with feature (`Translation`)[https://github.com/huggingface/datasets/blob/3edbeb0ec6519b79f1119adc251a1a6b379a2c12/src/datasets/features/translation.py#L8]

## Steps to reproduce the bug

```python
from datasets import load_dataset

dataset = load_dataset("europa_ecdc_tm", "en2fr", split="train[:10]")
print(dataset.features)
# {'translation': Translation(languages=['en', 'fr'], id=None)}
print(dataset[0])
# {'translation': {'en': 'Vaccination against hepatitis C is not yet available.', 'fr': 'Aucune vaccination contre l’hépatite C n’est encore disponible.'}}

dataset.flatten()
```

## Expected results

`dataset.flatten` should flatten the `Translation` column as if it were a dict of `Value("string")`

```python
dataset[0]
# {'translation.en':  'Vaccination against hepatitis C is not yet available.', 'translation.fr': 'Aucune vaccination contre l’hépatite C n’est encore disponible.' }
dataset.features
# {'translation.en': Value("string"), 'translation.fr': Value("string")}
```

## Actual results

```python
In [31]: dset.flatten()
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-31-bb88eb5276ee> in <module>
----> 1 dset.flatten()

[...]\site-packages\datasets\fingerprint.py in wrapper(*args, **kwargs)
    411             # Call actual function
    412
--> 413             out = func(self, *args, **kwargs)
    414
    415             # Update fingerprint of in-place transforms + update in-place history of transforms

[...]\site-packages\datasets\arrow_dataset.py in flatten(self, new_fingerprint, max_depth)
   1294                 break
   1295         dataset.info.features = self.features.flatten(max_depth=max_depth)
-> 1296         dataset._data = update_metadata_with_features(dataset._data, dataset.features)
   1297         logger.info(f'Flattened dataset from depth {depth} to depth {1 if depth + 1 < max_depth else "unknown"}.')
   1298         dataset._fingerprint = new_fingerprint

[...]\site-packages\datasets\arrow_dataset.py in update_metadata_with_features(table, features)
    534 def update_metadata_with_features(table: Table, features: Features):
    535     """To be used in dataset transforms that modify the features of the dataset, in order to update the features stored in the metadata of its schema."""
--> 536     features = Features({col_name: features[col_name] for col_name in table.column_names})
    537     if table.schema.metadata is None or b"huggingface" not in table.schema.metadata:
    538         pa_metadata = ArrowWriter._build_metadata(DatasetInfo(features=features))

[...]\site-packages\datasets\arrow_dataset.py in <dictcomp>(.0)
    534 def update_metadata_with_features(table: Table, features: Features):
    535     """To be used in dataset transforms that modify the features of the dataset, in order to update the features stored in the metadata of its schema."""
--> 536     features = Features({col_name: features[col_name] for col_name in table.column_names})
    537     if table.schema.metadata is None or b"huggingface" not in table.schema.metadata:
    538         pa_metadata = ArrowWriter._build_metadata(DatasetInfo(features=features))

KeyError: 'translation.en'
```

## Environment info

- `datasets` version: 1.18.3
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.7.10
- PyArrow version: 3.0.0