Array2D feature creates a list of list instead of a numpy array
Opened by offchan42 (closed)

### Describe the bug

I'm not sure if this is expected behavior or not. When I create a 2D array using `Array2D`, the data has list type instead of numpy array. I think it should not be the expected behavior especially when I feed a numpy array as input to the data creation function. Why is it converting my array into a list?
Also if I change the first dimension of the `Array2D` shape to None, it's returning array correctly.

### Steps to reproduce the bug

Run this code:
```py
from datasets import Dataset, Features, Array2D
import numpy as np

# you have to change the first dimension of the shape to None to make it return an array
features = Features(dict(seq=Array2D((2,2), 'float32')))  
ds = Dataset.from_dict(dict(seq=[np.random.rand(2,2)]), features=features)
a = ds[0]['seq']
print(a)
print(type(a))
```

The following will be printed in stdout:
```
[[0.8127174377441406, 0.3760348856449127], [0.7510159611701965, 0.4322739541530609]]
<class 'list'>
```

### Expected behavior

Each indexed item should be a list or numpy array.  Currently, `Array((2,2))` yields a list but `Array((None,2))` yields an array.

### Environment info

- `datasets` version: 2.11.0
- Platform: Windows-10-10.0.19045-SP0
- Python version: 3.9.13
- Huggingface_hub version: 0.13.4
- PyArrow version: 11.0.0
- Pandas version: 1.4.4


Comments:
Hi! 

You need to set the format to `np` before indexing the dataset to get NumPy arrays:
```python
features = Features(dict(seq=Array2D((2,2), 'float32')))  
ds = Dataset.from_dict(dict(seq=[np.random.rand(2,2)]), features=features)
ds.set_format("np")
a = ds[0]['seq']
```

>  I think it should not be the expected behavior especially when I feed a numpy array as input to the data creation function. Why is it converting my array into a list?

The same dataset can have examples in different types (Numpy arrays, Torch tensors, Pandas series, etc.), so recovering them all would be slow and impractical. Instead, the design of our formatting API is similar to Arrow's (the lib we use internally to store data on disk/ in RAM), which allows converting a batch of data to Python/Numpy/Pandas in a single call (and uses C++ to do so to make it faster).

> Also if I change the first dimension of the Array2D shape to None, it's returning array correctly.

Setting the first dimension to `None` makes it variable-length (allows passing arrays with the first dimensions of differing lengths).


Current behavior when indexing the dataset:
- Using `Array((2,2))` returns a list of lists.
- Using `Array((None,2))` returns a numpy array.

Don't you think this is kind of unexpected behavior from end-user perspective? 
As a user, I expect that when I use `Array2D`, the behavior needs to be consistent even if I specify None or not. It should either return a list or an array. It needs to choose one. Let's say if it always return a list, then I will call `ds.set_format('np')` no problem.

The consistency can be in any of these aspects:
1. preserves the type of the input data (in this case, a numpy array)
2. ensure the output type is always the same (it can be either list or array, but it needs to be one of them)

Right now the API doesn't conform to any of these aspects. But I think it needs to conform to one.

I thought we made this consistent by returning lists in both scenarios...

Fixed in #5751