SergeiGKS/huggingface-dataset-issues huggingface dataset:
Issue: Implement ability to define splits in metadata section of dataset card

Comment:
@merveenoyan @lhoestq I see two solutions to this case.  
1. Parse configurations automatically from directories names. That is, if you have data structure like:
```
tabular-benchmark
  └─clf_cat_compass
      └─compass.csv
  └─clf_cat_cat_covertype
      └─covertype.csv
 ...
  └─reg_cat_house_sales
      └─house_sales.csv
```
you'll get  "clf_cat_compass", "clf_cat_cat_covertype", ... "reg_cat_house_sales" configurations that would contain **only files from corresponding directories**. 
**\+** this is a requested change and needed in general and would solve other problems, see https://github.com/huggingface/datasets/issues/4578, would also help with https://github.com/huggingface/datasets/pull/5213 which I'm working on currently
**\+** would allow users to do just `load_dataset(“inria-soda/tabular-benchmark”, “clf_cat_compass”)`, no `data_files` param required
**\-** in this specific case it would require restructuring of the data - putting each file in a directory named as a config name (to me personally it doesn't seem to be a big deal) 

2. More or less what we discussed before - add support for manually specifying parameters in the metadata. We can add new metadata yaml field (say, `"custom_configs_info"`), so that we can provide smth like:
```yaml
---
...
dataset_info:
 ...  
custom_configs_info:
- config_name: reg_cat_house_sales
 data_files:
 - reg_cat/house_sales.csv
- config_name: clf_cat_compass
 data_files:
 - clf_cat/compass.csv
...
---
```
**\+** Would be useful not only for tabular data and not only for `data_files` parameter - any packaged dataset’s viewer can be customized to use specific, non-default parameters. @merveenoyan do you maybe have any other examples/use cases in mind where you want to provide any specific parameters to the viewer? 
**\-** I'm not sure here but assume that it might require changes in interaction with the viewer on the hub side - to parse these configurations, as they not default configurations (not in `BUILDER_CONFIGS` list). cc @severo But probably this can be solved on the `datasets` side too.

Overall, I would start from implementing the first solution since it's related to what I'm doing now and is super useful for `datasets` in general. And then  if we agree that having more flexibility in providing parameters to the viewer is required, I can implement the second one. Let me know what you think :)
Issue: Implement ability to define splits in metadata section of dataset card

Comment:
> We can add new metadata yaml field (say, "custom_configs_info"), so that we can provide smth like:

Love it ! Some other ideas to name the "custom_configs_info" field: "configs", "parameters", "config_args", "configurations"

> it might require changes in interaction with the viewer on the hub side - to parse these configurations, as they not default configurations (not in BUILDER_CONFIGS list)

If we update the `get_dataset_config_names()` function in `datasets` in inspect.py we should be fine - that's what the viewer is using

> Overall, I would start from implementing the first solution since it's related to what I'm doing now and is super useful for datasets in general. And then if we agree that having more flexibility in providing parameters to the viewer is required, I can implement the second one. Let me know what you think :)

Actually I feel like the second solution includes the first use case you mentioned. If you implement the second solution, then users would just have to add a few lines of YAML and their directories would be considered configurations no ? Maybe there's no need to implement two different logics to do the same thing
Issue: Implement ability to define splits in metadata section of dataset card

Comment:
@merveenoyan I haven't started working on this yet, working on adding configs to packaged datasets instead: https://github.com/huggingface/datasets/pull/5213 because this both would allow you to solve your issue and is a frequently requested feature.

adding arbitrary parameters to yaml would be my next task i think!
Issue: Connection error of the HuggingFace's dataset Hub due to SSLError with proxy

Comment:
Hi ! It looks like an issue with your python environment, can you make sure you're able to run GET requests to https://huggingface.co using `requests` in python ?
Issue: Connection error of the HuggingFace's dataset Hub due to SSLError with proxy

Comment:
Thanks for your reply. Does this mean that I have to use the `do_dataset `function and the `requests `function to download the dataset from the company's proxy environment?


* Reference: 
```bash
### How to load this dataset directly with the [datasets](https://github.com/huggingface/datasets) library


* https://huggingface.co/datasets/moyix/debian_csrc

* from datasets import load_dataset
dataset = load_dataset("moyix/debian_csrc")



### Or just clone the dataset repo


git lfs install
git clone https://huggingface.co/datasets/moyix/debian_csrc
# if you want to clone without large files – just their pointers
# prepend your git clone with the following env var:
GIT_LFS_SKIP_SMUDGE=1
```
Issue: Connection error of the HuggingFace's dataset Hub due to SSLError with proxy

Comment:
You can use `requests` to see if downloading a file from the Hugging Face Hub works. If so, then `datasets` should work as well. If not, then you have to find another way using an internet connection that works
Issue: Connection error of the HuggingFace's dataset Hub due to SSLError with proxy

Comment:
I resolved this issue by applying to "unblock websites" at https://huggingface.com in a corporate network environment with a firewall.
Issue: Use logging instead of printing to console

Comment:
Actually upon closer inspection, it is documented in the code that this behavior is intentional, so I'll close this.
Issue: Some links to canonical datasets in the docs are outdated

Comment:
Thanks for catching this, I can go through the docs and replace the links to their corresponding datasets on the Hub!
Issue: "One or several metadata. were found, but not in the same directory or in a parent directory"

Comment:
Also unrelated but still: https://huggingface.co/docs/datasets/image_dataset#generate-the-dataset
```If your loading script passed the test, you should now have a dataset_infos.json file in your dataset folder.```
It's not the case anymore as it's now in the readme.md, it was confusing to me
Issue: "One or several metadata. were found, but not in the same directory or in a parent directory"

Comment:
And here is my data loader script: https://huggingface.co/datasets/corentinm7/MyoQuant-SDH-Data/blob/main/SDH_16k.py
I have one file archive to download that contains the images for all splits and one `metadata.jsonl` to download that contains the informations about what image goes into what split.
Issue: "One or several metadata. were found, but not in the same directory or in a parent directory"

Comment:
Hi @lambda-science! It seems that your repo is recognized as a packaged module [ImageFolder](https://huggingface.co/docs/datasets/main/en/image_dataset#imagefolder), not as a dataset with the custom loading script, because loader looks for a script that has the same name as the dataset repo. So please try to rename your script to `MyoQuant-SDH-Data.py`, this should help.
Issue: "One or several metadata. were found, but not in the same directory or in a parent directory"

Comment:
> Hi @lambda-science! It seems that your repo is recognized as a packaged module [ImageFolder](https://huggingface.co/docs/datasets/main/en/image_dataset#imagefolder), not as a dataset with the custom loading script, because loader looks for a script that has the same name as the dataset repo. So please try to rename your script to `MyoQuant-SDH-Data.py`, this should help.

Hi !

Thank you for your answer. That was... embarrassingly easy, sorry for this issue, everything is fixed now ! 

Have a nice day ! :)
Issue: "One or several metadata. were found, but not in the same directory or in a parent directory"

Comment:
@lambda-science that's not embarrassing at all! it's actually not clear from the documentation that the script should have the same name, so thank you for the issue, we'll add this information to the docs :)
Issue: `path` is `None` when downloading a custom audio dataset from the Hub

Comment:
Hi! Yes, this is expected behavior - we do this as a security measure to not leak local paths (this info would be useless on other users' machines anyways) and only push audio bytes.
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
I have to admit I'm not a fan of this idea, as this would result in a non-consistent behavior between tabular and non-tabular datasets, which is confusing if done without the context you provided. Instead, we could consider returning a `Dataset` object rather than `DatasetDict` if there is only one split in the generated dataset. But then again, I think this lib is a bit too old to make such changes. @lhoestq @albertvillanova WDYT?
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
We can brainstorm here to see how we could make it happen ? And then depending on the options we see if it's a change we can do.

I'm starting with a first reasoning

Currently not passing `split=` in `load_dataset` means "return a dict with each split".

Now what would happen if a dataset has no split ? Ideally it should return one Dataset. And passing `split=` would have no sense. So depending on the dataset content, not passing `split=` should return a dict or a Dataset. In particular, those two cases should work:
```python
# case 1: dataset without split
ds = load_dataset("dataset_without_split")
ds[0], ds["column_name"], list(ds) # we want this

# case 2: dataset with splits
ds = load_dataset("dataset_with_splits")
ds["train"] # this works and can't be changed
ds = load_dataset("dataset_with_splits", split="train")
ds[0], ds["column_name"], list(ds) # this works and can't be changed
```

I can see several ideas:
1. allowing `load_dataset` to return a different object based on the dataset content - either a Dataset or a DatasetDict
    - we can update `get_dataset_split_names` to return None or a list if users want to know in advance what object will be returned. They can also use `isinstance` _a posteriori_
    - but in this case we expect users to be careful when loading datasets and always to extra steps to check if they got a Dataset or DatasetDict
2. merge Dataset and DatasetDict objects
    - they already share many functions: map, filter, push_to_hub etc.
    - we can define `ds[0]` to be the first item of the first split, and consider that the uses accesses rows from the full table of all the splits concatenated
    - however there is a collision when doing `ds["column_name"]` or `ds["train"]` that we need to address: the first returns a list, while the other returns a Dataset.

What are your opinions on those two ideas ? Do you have other ideas in mind ?
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
I like the first idea more (concatenating splits doesn't seem useful, no?). This is a significant breaking change, so I think we should do a poll (or something similar) to gather more info on the actual "expected behavior" and wait for Datasets 3.0 if we decide to implement it.

PS: @thomwolf also suggested the same thing a while ago (https://github.com/huggingface/datasets/issues/743#issuecomment-746074641).
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
I think it's an interesting improvement to the user experience for a case that comes often (no split) so I would definitively support it.

I would be more in favor of option 2 rather than returning various types of objects from load_dataset and handling carefully the possible collisions indeed
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
Related: if a dataset only has one split, we don't show the splits select control in the dataset viewer on the Hub, eg. compare  https://huggingface.co/datasets/hf-internal-testing/fixtures_image_utils/viewer/image/test with https://huggingface.co/datasets/glue/viewer/mnli/test.

See https://github.com/huggingface/moon-landing/pull/3858 for more details (internal)
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
I feel like the second idea is a bit more overkill. 
@severo I would say it's a bit irrelevant to the problem we have but is a separate problem @polinaeterna is solving at the moment. 😅 (also discussed on slack)
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
OK, sorry for polluting the thread. The relation I saw with the dataset viewer is that from a UX point of view, we hide the concepts of split and configuration whenever possible -> this issue feels like doing the same in the datasets library.
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
I would agree that returning different types based on the content of the dataset might be confusing.

We can do something similar to what `fetch_*` or `load_*` from `sklearn.datasets` do, which is to have an arg which changes the type of the returned type. For instance, `load_iris` would return a dict, but `load_iris(..., return_X_y=True)` would return a tuple.

Here we can have a similar arg such as `return_X` which would then only return a single `DataSet` or an array.
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
> I feel like the second idea is a bit more overkill.

Overkill in what sense ?

> Here we can have a similar arg such as return_X which would then only return a single DataSet or an array.

Right now one can already pass `split="all"` to get one `Dataset` object with all the data in it (unsplit). We could also have something like `return_all=True` so make the API clearer.

> I would be more in favor of option 2 rather than returning various types of objects from load_dataset and handling carefully the possible collisions indeed

I think it would be ok to handle the collision by allowing both `ds["train"]` and `ds["column_name"]` (and maybe adding something like `ds.splits` for those who want to iterate over the splits or add new ones)
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
Would it make sense to remove the notion of "split" in `load_dataset`? I feel a lof of it comes from the want to have some sort of group of more or less similar dataset. "train"/"test"/"validation" are the traditional ones, but there are some datasets that have much more splits.

Would it make sense to force `load_dataset` to only load a single `Dataset` object, and fail if it doesn't point to one. And have another method that's like `load_dataset_group_info` that can return a very arbitrary info class (Dict, List whatever), but you need to pass individual infos to `load_dataset` to run anything? Typically I don't think `DatasetDict.map` is really that helpful, but that's my personal opinion. This would help make things more readable (typically knowing if an object is a `Dataset` or a `DatasetDict`)
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
> Would it make sense to remove the notion of "split" in load_dataset?

I think we need to keep it - though in practice people can name the splits whatever they want anyway.

> Would it make sense to force load_dataset to only load a single Dataset object, and fail if it doesn't point to one.

We need to keep backward compatibility ideally - in particular the load_dataset + ds["train"] one
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
> I think we need to keep it - though in practice people can name the splits whatever they want anyway.

It was my understanding that the whole issue was that `load_dataset` returned multiple types of objects.

> We need to keep backward compatibility ideally - in particular the load_dataset + ds["train"] one

Yeah sorry I meant ideally. One can always start developing `load_dataset_v2` can deprecate the first one and remove it in the longer term.
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
> It was my understanding that the whole issue was that load_dataset returned multiple types of objects.

Yes indeed, but we still want to keep a way to load the train/val/test/whatever splits alone ;)
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
Started to experiment with merging Dataset and DatasetDict. My plan is to define the splits of a Dataset in Dataset.info.splits (already exists, but never used). A Dataset would then be the concatenation of its splits if they exist.

Not sure yet this is the way to go. My plan is to play with it and see and share it with you, so we can see if it makes sense from a UX point of view.
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
So just to make sure that I understand the current direction, people will have to be extra careful when handling splits right?
Imagine "potato" a dataset containing train/validation split:
```
load_dataset("potato") # returns the concatenation of all the splits
```
Previously the design would force you to choose a split (it would raise otherwise), or manually concat them if you really wanted to play with concatenated splits. Now it would potentially run without raising for a bit of time until you figure out that you've been training on both train and validation split.

Would it make sense to use a dataset specific default instead of using the concatenation, typically "potato" dataset's default would be train?
```
load_dataset("potato") # returns "train" split
load_dataset("potato", split="train") # returns "train" split
load_dataset("potato", split="validation") # returns "validation" split
concatenate_datasets([load_dataset("potato", split="train"), load_dataset("potato", split="validation")]) # returns concatenation
```
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
> load_dataset("potato") # returns "train" split

To avoid a breaking change we need to be able to do `load_dataset("potato")["validation"]` as well.

In that case I'd wonder where the validation split comes from, since the rows of the dataset wouldn't contain the validation split according to your example. That's why I'm more in favor of concatenating.

A dataset is one table, that optionally has some split info about subsets (e.g. for training an evaluation)

This also allows anyone to re-split the dataset the way they want if they're not happy with the default:

```python
ds = load_dataset("potato").train_test_split(test_size=0.2)
train_ds = ds["train"]
test_ds = ds["test"]
```
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
Just thinking about this, we could just have `to_dataframe()` as `load_dataset("blah").to_dataframe()` to get the whole dataset, and not change anything else.
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
I have a first implementation of option 2 (merging Dataset and DatasetDict) in this PR: https://github.com/huggingface/datasets/pull/5301/

Feel free to play with it if you're interested, and let me know what you think. In this PR, a dataset is one table that optionally has some split info about subsets.
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
@adrinjalali we already have [to_pandas](https://huggingface.co/docs/datasets/package_reference/main_classes#datasets.Dataset.to_pandas) AFAIK that essentially does the same thing (for a dataset, not for a dataset dict), I was wondering if it makes sense to have this as I don't know portion of people who load non-tabular datasets into dataframes. @lhoestq I saw your PR and it will break a lot of things imo, WDYT of this option?
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
> we already have [to_pandas](https://huggingface.co/docs/datasets/package_reference/main_classes#datasets.Dataset.to_pandas) AFAIK that essentially does the same thing (for a dataset, not for a dataset dict)

yes correct :)

> I saw your PR and it will break a lot of things imo

Do you have concrete examples you can share ?

> WDYT of this option?

The to_dataframe option ? I think it not enough, since you'd still get a `DatasetDict({"train": Dataset()})` if you load a dataset with no splits (e.g. one CSV), and this doesn't really make sense.

Note that in the PR I opened you can do
```python
ds = load_dataset("dataset_with_just_one_csv")  # Dataset type
df = load_dataset("dataset_with_just_one_csv").to_pandas()  # DataFrame type
```
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
@lhoestq no I think @adrinjalali and I meant when user calls `to_dataframe` if there's only train split in `DatasetDict` we could directly load that into dataframe. This might cause a confusion given there's to_pandas but I think it's more intuitive and least breaking change. (given people -who use `datasets` for tabular workflows- will eventually call `to_pandas` anyway)
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
So in that case it would be fine to still end up with a dataset dict with a "train" split ?
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
yeah what I mean is this:

```py
dataset = load_dataset("blah")

# deal with a split of the dataset
train = dataset["train"]
train_df = dataset["train"].to_dataframe()

# deal with the whole dataset
dataset_df = dataset.to_dataframe()
```

So we do two things to improve tabular experience:
- allow datasets to have a single split
- add `to_dataframe` to the root dict level so that users can simply call `df = load_dataset("blah").to_dataframe()` and have it in their `pandas.DataFrame` object.
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
Ok ! Note that we already have `Dataset.to_pandas()` so for consistency I'd call it `DatasetDict.to_pandas()` as well, does it sound good to you ? This is something we can add pretty easily
Issue: Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded

Comment:
> So just to make sure that I understand the current direction, people will have to be extra careful when handling splits right?

We can raise an error if someone does `load_dataset(...)[0]` if the dataset is made of several splits, and return the first example if there's one or zero splits (i.e. when it's not ambiguous). Had this idea from the dicussions in #5312 WDYT @thomasw21 ?