Local and automatic tests fail
State: closed | Author: MarkusSagen (NONE)
Labels: name: bug

## Describe the bug
Running the tests from CircleCI on a PR or locally fails, even with no changes. Tests seem to fail on `test_metric_common.py`

## Steps to reproduce the bug
```shell
git clone https://huggingface/datasets.git
cd datasets 
```
```python
python -m pip install -e .
pytest 
```

## Expected results
All tests passing

## Actual results
```
tests/test_metric_common.py:91: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../.pyenv/versions/3.8.5/lib/python3.8/doctest.py:1336: in __run
    exec(compile(example.source, filename, "single",
<doctest datasets_modules.metrics.ter.c0cfb5adedac7eb15ffa47bba6a70fabd80f3eb906ee508abf5e1906285d1155.ter.Ter[3]>:1: in <module>
    ???
../datasets/src/datasets/metric.py:430: in compute
    output = self._compute(**inputs, **compute_kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Metric(name: "ter", features: {'predictions': Value(dtype='string', id='sequence'), 'references': Sequence(feature=Val...ences=references)
    >>> print(results)
    {'score': 0.0, 'num_edits': 0, 'ref_length': 6.5}
""", stored examples: 0)
predictions = ['hello there general kenobi', 'foo bar foobar']
references = [['hello there general kenobi', 'hello there !'], ['foo bar foobar', 'foo bar foobar']]
normalized = False, no_punct = False, asian_support = False, case_sensitive = False

    def _compute(
        self,
        predictions,
        references,
        normalized: bool = False,
        no_punct: bool = False,
        asian_support: bool = False,
        case_sensitive: bool = False,
    ):
        references_per_prediction = len(references[0])
        if any(len(refs) != references_per_prediction for refs in references):
            raise ValueError("Sacrebleu requires the same number of references for each prediction")
        transformed_references = [[refs[i] for refs in references] for i in range(references_per_prediction)]
    
>       sb_ter = TER(normalized, no_punct, asian_support, case_sensitive)
E       TypeError: __init__() takes 2 positional arguments but 5 were given

/tmp/pytest-of-markussagen/pytest-1/cache/modules/datasets_modules/metrics/ter/c0cfb5adedac7eb15ffa47bba6a70fabd80f3eb906ee508abf5e1906285d1155/ter.py:130: TypeError
------------------------------ Captured stdout call -------------------------------
Trying:
    predictions = ["hello there general kenobi", "foo bar foobar"]
Expecting nothing
ok
Trying:
    references = [["hello there general kenobi", "hello there !"], ["foo bar foobar", "foo bar foobar"]]
Expecting nothing
ok
Trying:
    ter = datasets.load_metric("ter")
Expecting nothing
ok
Trying:
    results = ter.compute(predictions=predictions, references=references)
Expecting nothing
================================ warnings summary =================================
../.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/hdfs/config.py:15
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/hdfs/config.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    from imp import load_source

../datasets/src/datasets/commands/test.py:35
  /home/markussagen/datasets/src/datasets/commands/test.py:35: PytestCollectionWarning: cannot collect test class 'TestCommand' because it has a __init__ constructor (from: tests/commands/test_test.py)
    class TestCommand(BaseDatasetsCLICommand):

tests/commands/test_test.py:33
  /home/markussagen/mydataset/tests/commands/test_test.py:33: PytestCollectionWarning: cannot collect test class 'TestCommandArgs' because it has a __new__ constructor (from: tests/commands/test_test.py)
    class TestCommandArgs:

tests/test_arrow_dataset.py: 760 warnings
tests/test_formatting.py: 60 warnings
tests/test_search.py: 31 warnings
tests/features/test_array_xd.py: 117 warnings
  /home/markussagen/datasets/src/datasets/formatting/formatting.py:197: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. 
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    (isinstance(x, np.ndarray) and (x.dtype == np.object or x.shape != array[0].shape))

tests/test_arrow_dataset.py: 154 warnings
tests/features/test_array_xd.py: 1 warning
  /home/markussagen/datasets/src/datasets/formatting/formatting.py:201: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. 
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    return np.array(array, copy=False, **{**self.np_array_kwargs, "dtype": np.object})

tests/test_arrow_dataset.py: 60 warnings
  /home/markussagen/datasets/src/datasets/arrow_dataset.py:3105: DeprecationWarning: `np.str` is a deprecated alias for the builtin `str`. To silence this warning, use `str` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.str_` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    elif np.issubdtype(values.dtype, np.str):

tests/test_arrow_dataset.py: 138 warnings
tests/test_formatting.py: 21 warnings
  /home/markussagen/datasets/src/datasets/formatting/tf_formatter.py:69: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. 
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    data_struct.dtype == np.object

tests/test_arrow_dataset.py: 240 warnings
tests/test_formatting.py: 20 warnings
  /home/markussagen/datasets/src/datasets/formatting/torch_formatter.py:49: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. 
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    if data_struct.dtype == np.object:  # pytorch tensors cannot be instantied from an array of objects

tests/test_arrow_dataset.py: 12 warnings
tests/test_search.py: 2 warnings
tests/features/test_array_xd.py: 6 warnings
tests/features/test_image.py: 4 warnings
  /home/markussagen/datasets/src/datasets/features/features.py:1129: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. 
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    [0] + [len(arr) for arr in l_arr], dtype=np.object

tests/test_dataset_common.py::LocalDatasetTest::test_builder_class_banking77
  /tmp/pytest-of-markussagen/pytest-1/cache/modules/datasets_modules/datasets/banking77/aec0289529599d4572d76ab00c8944cb84f88410ad0c9e7da26189d31f62a55b/banking77.py:24: DeprecationWarning: invalid escape sequence \~
    _CITATION = """\

tests/test_dataset_common.py::LocalDatasetTest::test_builder_class_universal_dependencies
  /tmp/pytest-of-markussagen/pytest-1/cache/modules/datasets_modules/datasets/universal_dependencies/065e728dfe9a8371434a6e87132c2386a6eacab1a076d3a12aa417b994e6ef7d/universal_dependencies.py:6: DeprecationWarning: invalid escape sequence \=
    _CITATION = """\

tests/test_filesystem.py: 105 warnings
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/responses/__init__.py:398: DeprecationWarning: stream argument is deprecated. Use stream parameter in request directly
    warn(

tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter_np_array_kwargs
tests/test_formatting.py::FormatterTest::test_jax_formatter_np_array_kwargs
tests/test_formatting.py::FormatterTest::test_jax_formatter_np_array_kwargs
tests/test_formatting.py::FormatterTest::test_jax_formatter_np_array_kwargs
  /home/markussagen/datasets/src/datasets/formatting/jax_formatter.py:57: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. 
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    if data_struct.dtype == np.object:  # jax arrays cannot be instantied from an array of objects

tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/jax/_src/numpy/lax_numpy.py:3567: UserWarning: Explicitly requested dtype <class 'jax._src.numpy.lax_numpy.int64'> requested in array is not available, and will be truncated to dtype int32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.
    lax._check_user_dtype_supported(dtype, "array")

tests/test_metric_common.py::LocalMetricTest::test_load_metric_frugalscore
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/apscheduler/util.py:95: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
    if obj.zone == 'local':

tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_to_hub_custom_features
_audio
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/librosa/core/constantq.py:1059: DeprecationWarning: `np.complex` is a deprecated alias for the builtin `complex`. To silence this warning, use `complex` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.complex128` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    dtype=np.complex,

tests/features/test_array_xd.py::test_array_xd_with_none
  /home/markussagen/mydataset/tests/features/test_array_xd.py:338: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. 
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    assert isinstance(arr, np.ndarray) and arr.dtype == np.object and arr.shape == (3,)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
============================= short test summary info =============================
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_bleurt - I...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_chrf - Att...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_code_eval
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_comet - Im...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_competition_math
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_coval - Im...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_frugalscore
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_perplexity
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_ter - Type...
```

## Environment info
- `datasets` version: 2.0.1.dev0
- Platform: Linux-5.16.11-76051611-generic-x86_64-with-glibc2.33
- Python version: 3.8.5
- PyArrow version: 5.0.0


Comments: Hi ! To be able to run the tests, you need to install all the test dependencies and additional ones with
```
pip install -e .[tests]
pip install -r additional-tests-requirements.txt --no-deps
```

In particular, you probably need to `sacrebleu`. It looks like it wasn't able to instantiate `sacrebleu.TER` properly.