Dataset loader for the MSLR2022 shared task
Opened by JohnGiorgi (closed)

This PR adds a dataset loader for the [MSLR2022 Shared Task](https://github.com/allenai/mslr-shared-task). Both the MS^2 and Cochrane datasets can be loaded with this dataloader:

```python
from datasets import load_dataset

ms2 = load_dataset("mslr2022", "ms2")
cochrane = load_dataset("mslr2022", "cochrane")
```

Usage looks like:

```python
>>> ms2 = load_dataset("mslr2022", "ms2", split="validation")
>>> ms2.keys()
dict_keys(['review_id', 'pmid', 'title', 'abstract', 'target', 'background', 'reviews_info'])
>>> ms2[0].target
'Conclusions SC therapy is effective for PAH in pre clinical studies .\nThese results may help to st and ardise pre clinical animal studies and provide a theoretical basis for clinical trial design in the future .'
```

I have tested this works with the following command:

```bash
datasets-cli test datasets/mslr2022 --save_infos --all_configs
```

However I have having a little trouble generating the dummy data

```bash
datasets-cli dummy_data datasets/mslr2022 --auto_generate
```

errors out with the following stack trace:

```
Couldn't generate dummy file 'datasets/mslr2022/dummy/ms2/1.0.0/dummy_data/mslr_data.tar.gz/mslr_data/ms2/convert_to_cochrane.py'. Ignore that if this file is not useful for dummy data.
Traceback (most recent call last):                                                                 
  File "/Users/johngiorgi/.pyenv/versions/datasets/bin/datasets-cli", line 11, in <module>
    load_entry_point('datasets', 'console_scripts', 'datasets-cli')()
  File "/Users/johngiorgi/Documents/dev/datasets/src/datasets/commands/datasets_cli.py", line 39, in main
    service.run()
  File "/Users/johngiorgi/Documents/dev/datasets/src/datasets/commands/dummy_data.py", line 319, in run
    keep_uncompressed=self._keep_uncompressed,
  File "/Users/johngiorgi/Documents/dev/datasets/src/datasets/commands/dummy_data.py", line 361, in _autogenerate_dummy_data
    dataset_builder._prepare_split(split_generator, check_duplicate_keys=False)
  File "/Users/johngiorgi/Documents/dev/datasets/src/datasets/builder.py", line 1146, in _prepare_split
    desc=f"Generating {split_info.name} split",
  File "/Users/johngiorgi/.pyenv/versions/3.7.13/envs/datasets/lib/python3.7/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/Users/johngiorgi/.cache/huggingface/modules/datasets_modules/datasets/mslr2022/b4becd2f52cf18255d4934d7154c2a1127fb393371b87b3c1fc2c8b35a777cea/mslr2022.py", line 149, in _generate_examples
    reviews_info_df = pd.read_csv(reviews_info_filepath, index_col=0)
  File "/Users/johngiorgi/.pyenv/versions/3.7.13/envs/datasets/lib/python3.7/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/Users/johngiorgi/.pyenv/versions/3.7.13/envs/datasets/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/Users/johngiorgi/.pyenv/versions/3.7.13/envs/datasets/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 488, in _read
    return parser.read(nrows)
  File "/Users/johngiorgi/.pyenv/versions/3.7.13/envs/datasets/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 1047, in read
    index, columns, col_dict = self._engine.read(nrows)
  File "/Users/johngiorgi/.pyenv/versions/3.7.13/envs/datasets/lib/python3.7/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 224, in read
    chunks = self._reader.read_low_memory(nrows)
  File "pandas/_libs/parsers.pyx", line 801, in pandas._libs.parsers.TextReader.read_low_memory
  File "pandas/_libs/parsers.pyx", line 857, in pandas._libs.parsers.TextReader._read_rows
  File "pandas/_libs/parsers.pyx", line 843, in pandas._libs.parsers.TextReader._tokenize_rows
  File "pandas/_libs/parsers.pyx", line 1925, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: EOF inside string starting at row 2
```

I think this may have to do with unusual line terminators in the original data. When I open it in VSCode, it complains:

```
The file 'dev-inputs.csv' contains one or more unusual line terminator characters, like Line Separator (LS) or Paragraph Separator (PS).

It is recommended to remove them from the file. This can be configured via `editor.unusualLineTerminators`.
```

Tagging the organizers of the shared task in case they want to sanity check this or add any info to the model card :) @lucylw  @jayded



Comments:
I think the underlying issue is in https://github.com/huggingface/datasets/blob/c0ed6fdc29675b3565b01b77fde5ab5d9d8b60ec/src/datasets/commands/dummy_data.py#L124 - where `CSV`s are considered to be in the same class of file as text, jsonl, and tsv.

I think this is an error because CSVs can have newlines within the rows of a file. I'm happy to make a PR to change how this handling works, or make the change within this PR. 

We should figure out:
1. Does this dummy data need to be generated more than once? (It looks like no)
2. Should this be fixed generally? (needs a HF person to weigh in here)
3. What is the right way for such a fix to exist permanently here; the [Contributing document](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md) doesn't provide guidance on any tests. Writing a test is several times more effort than fixing the underlying issue. (again needs a HF person)

Would someone from HF mind taking a look at this PR? (@lhoestq)

Hi ! Sorry for the delay in responding :)

I don't think there's a big need to fix this in the general case for now, feel free to just generate the dummy data for this specific dataset :)

The `datasets-cli dummy_data datasets/mslr2022` command should tell you what dummy files to generate. In each dummy file you just need to include enough data to generate 4 or 5 examples

_The documentation is not available anymore as the PR was closed or merged._

Awesome! Generated the dummy data and the tests now pass. @jayded thanks for your help! If you and @lucylw are happy with this I think it's ready to be merged. @lhoestq this is ready for another look :)

Hi @lhoestq, is there anything blocking this from being merged that I can address?

Hi @JohnGiorgi ! Thanks for the changes, it looks all good now :)

I think this dataset can be under the AllenAI page here: https://huggingface.co/allenai What do you think ?
Feel free to create a new dataset repository on huggingface.co and upload your files (dataset script, readme, etc.)

Once the dataset is under the AllenAI org, we can close this PR


> Hi @JohnGiorgi ! Thanks for the changes, it looks all good now :)
> 
> I think this dataset can be under the AllenAI page here: https://huggingface.co/allenai What do you think ? Feel free to create a new dataset repository on huggingface.co and upload your files (dataset script, readme, etc.)
> 
> Once the dataset is under the AllenAI org, we can close this PR

Sweet! It is uploaded here: https://huggingface.co/datasets/allenai/mslr2022

Nice ! Thanks :)

I think we can close this PR then.

I noticed that the dataset preview is not available on this dataset, this is because we require datasets to work in streaming mode to show a preview. However TAR archives don't work well in streaming mode (you can't know in advance what files are inside a TAR archive without reading it completely). This can be fixed by using a ZIP archive instead.

Let me know if you have questions or if I can help.