KMFODA (CONTRIBUTOR) commented on 2020-12-21 13:46:06:
HI @SBrandeis. Thanks for all the comments - very helpful. I realised that the tests had failed and had been trying to figure out what was causing them to do so. All the tests pass when I run the load_real_dataset test however when I run `RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_persian_ner` I get the below error. One thing to note is that the automated dummy data file generation failed when I tried to run it so I manually created the dummy data and ensured that the last line in the file was an empty line as per your comments. Would appreciate your thoughts on what might be causing this:

```
__________________________________________________ LocalDatasetTest.test_load_dataset_all_configs_persian_ner __________________________________________________

self = <tests.test_dataset_common.LocalDatasetTest testMethod=test_load_dataset_all_configs_persian_ner>, dataset_name = 'persian_ner'

    @slow
    def test_load_dataset_all_configs(self, dataset_name):
        configs = self.dataset_tester.load_all_configs(dataset_name, is_local=True)
>       self.dataset_tester.check_load_dataset(dataset_name, configs, is_local=True)

tests/test_dataset_common.py:237: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_dataset_common.py:198: in check_load_dataset
    self.parent.assertTrue(len(dataset[split]) > 0)
E   AssertionError: False is not true
--------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------
Downloading and preparing dataset persian_ner/fold1 (download: 1.00 MiB, generated: 1.00 MiB, post-processed: Unknown size, total: 2.00 MiB) to /var/folders/nk/yp5_m5c95cnc0cm_vbd7h7g80000gn/T/tmpzh495aac/persian_ner/fold1/1.1.0...
Dataset persian_ner downloaded and prepared to /var/folders/nk/yp5_m5c95cnc0cm_vbd7h7g80000gn/T/tmpzh495aac/persian_ner/fold1/1.1.0. Subsequent calls will reuse this data.
--------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------
                                
======================================================================= warnings summary =======================================================================
env/lib/python3.7/site-packages/tensorflow/python/autograph/utils/testing.py:21
  /Users/karimfoda/Documents/STUDIES/PYTHON/DATASETS/env/lib/python3.7/site-packages/tensorflow/python/autograph/utils/testing.py:21: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

env/lib/python3.7/site-packages/apache_beam/typehints/typehints.py:693
  /Users/karimfoda/Documents/STUDIES/PYTHON/DATASETS/env/lib/python3.7/site-packages/apache_beam/typehints/typehints.py:693: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    if not isinstance(type_params, collections.Iterable):

env/lib/python3.7/site-packages/apache_beam/typehints/typehints.py:532
  /Users/karimfoda/Documents/STUDIES/PYTHON/DATASETS/env/lib/python3.7/site-packages/apache_beam/typehints/typehints.py:532: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    if not isinstance(type_params, (collections.Sequence, set)):

env/lib/python3.7/site-packages/elasticsearch/compat.py:38
  /Users/karimfoda/Documents/STUDIES/PYTHON/DATASETS/env/lib/python3.7/site-packages/elasticsearch/compat.py:38: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import Mapping

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=================================================================== short test summary info ====================================================================
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_persian_ner - AssertionError: False is not true
```