Logiqa en

logiqa in english.

Labels: name: dataset contribution

I'm getting an error when I try to create the dummy data:
```python
aclifton@pop-os:~/data/hf_datasets_sprint/datasets$ python datasets-cli dummy_data ./datasets/logiqa_en/ --auto_generate 
2021-01-07 10:50:12.024791: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-01-07 10:50:12.024814: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Using custom data configuration default
Couldn't generate dummy file 'datasets/dummy/1.1.0/dummy_data/master.zip/LogiQA-dataset-master/README.md'. Ignore that if this file is not useful for dummy data.
Dummy data generation done but dummy data test failed since splits ['train', 'test', 'validation'] have 0 examples for config 'default''.
Automatic dummy data generation failed for some configs of './datasets/logiqa_en/'
```

Hi ! Sorry for the delay

To fix your issue for the dummy data you must increase the number of lines that will be kept to generate the dummy files. By default it's 5, and as you need at least 8 lines here to yield one example you must increase this.

You can increase the number of lines to 32 for example by doing
```
datasets-cli dummy_data ./datasets/logica_en --auto_generate --n_lines 32
```

Also it looks like there are changes about other datasets in this PR (imppres). Can you fix that ? You may need to create another branch and another PR.

To fix the branch issue, I went ahead and made a backup of the dataset then deleted my local copy of my fork of `datasets`. I then followed the [detailed guide](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md) from the beginning to reclone the fork and start a new branch. 

However, when it came time to create the dummy data I got the following error:
```python
aclifton@pop-os:~/data/hf_datasets_sprint/datasets$ datasets-cli dummy_data ./datasets/logiqa_en --auto_generate --n_lines 32
2021-02-03 11:23:23.145885: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-02-03 11:23:23.145914: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Using custom data configuration default
Couldn't generate dummy file 'datasets/logiqa_en/dummy/1.1.0/dummy_data/master.zip/LogiQA-dataset-master/README.md'. Ignore that if this file is not useful for dummy data.
Traceback (most recent call last):
  File "/home/aclifton/anaconda3/bin/datasets-cli", line 36, in <module>
    service.run()
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/commands/dummy_data.py", line 317, in run
    keep_uncompressed=self._keep_uncompressed,
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/commands/dummy_data.py", line 355, in _autogenerate_dummy_data
    dataset_builder._prepare_split(split_generator)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/builder.py", line 905, in _prepare_split
    example = self.info.features.encode_example(record)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/features.py", line 799, in encode_example
    return encode_nested_example(self, example)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/features.py", line 710, in encode_nested_example
    (k, encode_nested_example(sub_schema, sub_obj)) for k, (sub_schema, sub_obj) in utils.zip_dict(schema, obj)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/features.py", line 710, in <genexpr>
    (k, encode_nested_example(sub_schema, sub_obj)) for k, (sub_schema, sub_obj) in utils.zip_dict(schema, obj)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/features.py", line 737, in encode_nested_example
    return schema.encode_example(obj)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/features.py", line 522, in encode_example
    example_data = self.str2int(example_data)
  File "/home/aclifton/anaconda3/lib/python3.7/site-packages/datasets/features.py", line 481, in str2int
    output.append(self._str2int[str(value)])
KeyError: "Some Cantonese don't like chili, so some southerners don't like chili."
```

Hi ! The error happens when the script is verifying that the generated dummy data work fine with the dataset script.
Apparently it fails because the text `"Some Cantonese don't like chili, so some southerners don't like chili."` was given in a field that is a ClassLabel feature (probably the `answer` field), while it actually expects "a", "b", "c" or "d". Can you fix the script so that it returns the expected labels for this field instead of the text ?

Also it would be awesome to rename this field `answerKey` instead of `answer` to have the same column names as the other multiple-choice-QA datasets in the library :) 

Ok getting closer! I got the dummy data to work. However I am now getting the following error:
```python
aclifton@pop-os:~/data/hf_datasets_sprint/datasets$ RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_logiqa_en
===================================================================== test session starts ======================================================================
platform linux -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /home/aclifton/data/hf_datasets_sprint/datasets
plugins: astropy-header-0.1.2, xdist-2.1.0, doctestplus-0.5.0, forked-1.3.0, hypothesis-5.5.4, arraydiff-0.3, remotedata-0.3.2, openfiles-0.4.0
collected 0 items / 1 error                                                                                                                                    

============================================================================ ERRORS ============================================================================
________________________________________________________ ERROR collecting tests/test_dataset_common.py _________________________________________________________
ImportError while importing test module '/home/aclifton/data/hf_datasets_sprint/datasets/tests/test_dataset_common.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_dataset_common.py:42: in <module>
    from datasets.packaged_modules import _PACKAGED_DATASETS_MODULES
E   ModuleNotFoundError: No module named 'datasets.packaged_modules'
----------------------------------------------------------------------- Captured stderr ------------------------------------------------------------------------
2021-02-10 11:06:14.345510: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-02-10 11:06:14.345551: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
======================================================================= warnings summary =======================================================================
/home/aclifton/anaconda3/lib/python3.7/site-packages/tensorflow/python/autograph/utils/testing.py:21
  /home/aclifton/anaconda3/lib/python3.7/site-packages/tensorflow/python/autograph/utils/testing.py:21: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

/home/aclifton/anaconda3/lib/python3.7/site-packages/apache_beam/typehints/typehints.py:693
  /home/aclifton/anaconda3/lib/python3.7/site-packages/apache_beam/typehints/typehints.py:693: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    if not isinstance(type_params, collections.Iterable):

/home/aclifton/anaconda3/lib/python3.7/site-packages/apache_beam/typehints/typehints.py:532
  /home/aclifton/anaconda3/lib/python3.7/site-packages/apache_beam/typehints/typehints.py:532: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    if not isinstance(type_params, (collections.Sequence, set)):

/home/aclifton/anaconda3/lib/python3.7/site-packages/elasticsearch/compat.py:38
  /home/aclifton/anaconda3/lib/python3.7/site-packages/elasticsearch/compat.py:38: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    from collections import Mapping

-- Docs: https://docs.pytest.org/en/latest/warnings.html
================================================================= 4 warnings, 1 error in 2.74s =================================================================
ERROR: not found: /home/aclifton/data/hf_datasets_sprint/datasets/tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_logiqa_en
(no name '/home/aclifton/data/hf_datasets_sprint/datasets/tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_logiqa_en' in any of [<Module test_dataset_common.py>])

```

Hi ! It looks like the version of `datasets` that is installed in your environment doesn't match the version of `datasets` you're using for the tests. Can you try uninstalling datasets and reinstall it again ?
```
pip uninstall datasets -y
pip install -e .
```

Closer still!
```python
aclifton@pop-os:~/data/hf_datasets_sprint/datasets$ git commit
[logiqa_en 2664fe7f] fixed several issues with logiqa_en.
 4 files changed, 324 insertions(+)
 create mode 100644 datasets/logiqa_en/README.md
 create mode 100644 datasets/logiqa_en/dataset_infos.json
 create mode 100644 datasets/logiqa_en/dummy/1.1.0/dummy_data.zip
 create mode 100644 datasets/logiqa_en/logiqa_en.py
aclifton@pop-os:~/data/hf_datasets_sprint/datasets$ git fetch upstream
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), 590 bytes | 590.00 KiB/s, done.
From https://github.com/huggingface/datasets
   6e114a0c..318b09eb  master     -> upstream/master
aclifton@pop-os:~/data/hf_datasets_sprint/datasets$ git rebase upstream/master 
error: cannot rebase: You have unstaged changes.
error: Please commit or stash them.
aclifton@pop-os:~/data/hf_datasets_sprint/datasets$ git push -u origin logiqa_en
Username for 'https://github.com': aclifton314
Password for 'https://aclifton314@github.com': 
To https://github.com/aclifton314/datasets
 ! [rejected]          logiqa_en -> logiqa_en (non-fast-forward)
error: failed to push some refs to 'https://github.com/aclifton314/datasets'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
```

Thanks for your contribution, @aclifton314. Are you still interested in adding this dataset?

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

We would suggest you create this dataset there. Please, feel free to tell us if you need some help.