Repository: MouseLand/suite2p (language: python)

Issue:
Test data
Hi,

I'm trying to rebase on the `main` branch to fix the conflicts in https://github.com/MouseLand/suite2p/pull/840, but I'm having some issues with the test data:

1. for some reason it fails at downloading all the files that are bigger than 1.01GB. Does anyone also have the same problem?
2. if I download the data manually and put it in the `data/test_data` folder [as written in the README](https://github.com/MouseLand/suite2p#installation-for-developers), it still tries to download all the test data. Is it the correct folder to put the test data in?


Discussion / hints:
Hi @kir0ul, 
1. Yeah, we've noticed that sometimes the download will stop abruptly if the files are too big. Are you using a Mac machine or a windows machine? Which zip file does it seem to be failing on?
2. The README.md has not been updated to reflect this change in how we organize the test data. I'll go ahead and make the change to the README.md. You can find the test inputs [here](https://www.suite2p.org/static/test_data/test_inputs.zip)  and the test outputs [here](https://www.suite2p.org/static/test_data/test_outputs.zip).
1. I'm on Linux :penguin: Looking more closely, what seems to happen is that it downloads the `test_inputs.zip` without any problem, but systematically fails for the `test_outputs.zip` at 1.01GB, then it tries to download it again for each test:
```
❯ pytest -vs
======================================== test session starts ========================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 -- /home/apierre/miniconda3/envs/suite2p/bin/python
cachedir: .pytest_cache
collected 30 items

tests/test_io.py::test_h5_to_binary_produces_nonnegative_output_data
Downloading: https://www.suite2p.org/static/test_data/test_inputs.zip
100%|█████████████████████████████████████████████████████████████| 966M/966M [02:51<00:00, 5.90MB/s]

Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [03:25<00:42, 5.27MB/s]
ERROR
tests/test_io.py::test_that_bin_movie_without_badframes_results_in_a_same_size_array
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:48<00:35, 6.41MB/s]
ERROR
tests/test_io.py::test_that_bin_movie_with_badframes_results_in_a_smaller_array
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:47<00:34, 6.45MB/s]
ERROR
tests/test_io.py::test_that_binaryfile_data_is_repeatable
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:49<00:35, 6.38MB/s]
ERROR
tests/test_io.py::test_nwb_round_trip[1plane1chan] ERROR
tests/test_io.py::test_nwb_round_trip[1plane1chan1500] ERROR
tests/test_io.py::test_nwb_round_trip[2plane2chan] ERROR
tests/test_io.py::test_nwb_round_trip[2plane2chan1500] ERROR
tests/test_io.py::test_get_suite2p_path[D:/kjkcc/jodendopn/suite2p/ncconoc/onowcno-D:/kjkcc/jodendopn/suite2p-True] PASSED
tests/test_io.py::test_get_suite2p_path[/home/bla/kjkcc/jodendopn/suite2p/ops.npy-/home/bla/kjkcc/jodendopn/suite2p-True] PASSED
tests/test_io.py::test_get_suite2p_path[/etc/bla/kjkcc/jodendopn/suite2p-/etc/bla/kjkcc/jodendopn/suite2p-True] PASSED
tests/test_io.py::test_get_suite2p_path[/etc/bla/kjkcc/jodendopn/--False] PASSED
tests/test_registration.py::test_spatial_smooth_has_not_regressed_during_refactor PASSED
tests/test_registration.py::test_positive_bidiphase_shift_shifts_every_other_line PASSED
tests/test_registration.py::test_negative_bidiphase_shift_shifts_every_other_line PASSED
tests/regression/test_classification_pipeline.py::test_classification_output
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:31<00:31, 7.12MB/s]
ERROR
tests/regression/test_detection_pipeline.py::test_detection_output_1plane1chan
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [03:03<00:38, 5.91MB/s]
ERROR
tests/regression/test_detection_pipeline.py::test_detection_output_2plane2chan
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:31<00:31, 7.14MB/s]
ERROR
tests/regression/test_extraction_pipeline.py::test_pre_process_baseline
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [03:27<00:43, 5.23MB/s]
ERROR
tests/regression/test_extraction_pipeline.py::test_extraction_output_1plane1chan
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [03:43<00:46, 4.85MB/s]
ERROR
tests/regression/test_extraction_pipeline.py::test_extraction_output_2plane2chan
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:53<00:36, 6.24MB/s]
ERROR
tests/regression/test_full_pipeline.py::test_1plane_1chan_with_batches_metrics_and_exported_to_nwb_format
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [03:48<00:47, 4.73MB/s]
ERROR
tests/regression/test_full_pipeline.py::test_2plane_2chan_with_batches
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [03:07<00:39, 5.77MB/s]
ERROR
tests/regression/test_full_pipeline.py::test_mesoscan_2plane_2z
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:38<00:33, 6.81MB/s]
ERROR
tests/smoke/test_cli.py::test_cli_help_test_appears_when_suite2p_is_called PASSED
tests/smoke/test_cli.py::test_cli_help_test_appears_when_suite2p_is_called_locally PASSED
tests/smoke/test_cli.py::test_cli_help_test_appears_when_reg_metrics_is_called PASSED
tests/smoke/test_cli.py::test_cli_reg_metrics_runs_with_output PASSED
tests/smoke/test_file_loading.py::test_bruker
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:32<00:31, 7.11MB/s]
ERROR
tests/smoke/test_run_s2p.py::test_do_registration_do_roi_detect_settings_check_timing
Downloading: https://www.suite2p.org/static/test_data/test_outputs.zip
 83%|████████████████████████████████████████████████▊          | 1.01G/1.22G [02:23<00:29, 7.55MB/s]
ERROR
```
Even when I downloaded the test data manually I had to retry the download a few times, so maybe a retry decorator could be added to the downloading function? Some possible existing solutions for this could be [Tenacity](https://tenacity.readthedocs.io/), [urllib3](https://urllib3.readthedocs.io/en/stable/user-guide.html#retrying-requests), or [backoff](https://github.com/litl/backoff).

2. Thanks, it works now!

Out of curiosity, may I ask why you're not using DVC anymore? (We're maybe considering to use it on a project  but wonder if there are any drawbacks).
1. Ah, that's a great point. I'll add a retry decorator for the downloading function.
2. I think we swapped to this approach due to its simplicity. With this .zip method, it's much easier to just host the data on some site. We found it easier to update the test data inputs and outputs (e.g., if we regenerate the expected test data outputs, we can just drag and drop it to our site). There's no need to learn any dvc commands or how the .dvc files work.

Seems this issue's been fixed though so I'll close it for now. Feel free to comment on it though if you'd like to discuss anything else!

Fix patch:
diff --git a/docs/installation.rst b/docs/installation.rst
index 5cacba6c..115754f0 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -1,33 +1,7 @@
 Installation
 ----------------
 
-Install an `Anaconda`_ distribution of Python -- Choose **Python 3.x**
-and your operating system. Note you might need to use an anaconda prompt
-if you did not add anaconda to the path.
-
-1. Download the suite2p repository from GitHub using Git:  ``git clone https://github.com/MouseLand/suite2p``
-2. Open an anaconda prompt / command prompt with ``conda`` for **python
-   3** in the path
-3. Change the current directory to the suite2p folder: ``cd suite2p``
-4. Run ``conda env create -f environment.yml``
-5. To activate this new environment, run ``conda activate suite2p``. Afterwards, You should see ``(suite2p)`` on the left side of the terminal line.
-6. Install suite2p into this environment: ``pip install suite2p``
-7. Now run ``suite2p`` and you're all set.
-
-If you have an older ``suite2p`` environment you can remove it with
-``conda env remove -n suite2p`` before creating a new one.
-
-Note you will always have to run **conda activate suite2p** before you
-run suite2p. Conda ensures mkl_fft and numba run correctly and quickly
-on your machine. If you want to run jupyter notebooks in this
-environment, then also ``conda install jupyter``.
-
-To upgrade suite2p (package `here`_), run the following in the
-environment:
-
-::
-
-   pip install suite2p --upgrade
+Please refer to the suite2p `README`_ for the latest up-to-date installation instructions.
 
 **Common issues**
 
@@ -93,3 +67,4 @@ Dependencies
 .. _Anaconda: https://www.anaconda.com/download/
 .. _here: https://pypi.org/project/suite2p/
 .. _link: https://askubuntu.com/questions/1406379/qt5-install-problem-ubuntustudio-22-04/1406503#1406503
+.. _README: https://github.com/MouseLand/suite2p#readme
\ No newline at end of file
diff --git a/setup.py b/setup.py
index 092849c6..c51fdc6b 100644
--- a/setup.py
+++ b/setup.py
@@ -29,6 +29,7 @@
       ]
 test_deps = [
       'pytest',
+      'tenacity',
       'tqdm',
       'pytest-qt==3.3.0',
     ]


Test patch:
diff --git a/conftest.py b/conftest.py
index 062ba80c..68f95599 100644
--- a/conftest.py
+++ b/conftest.py
@@ -7,6 +7,7 @@
 from tqdm import tqdm
 from pathlib import Path
 from urllib.request import urlopen
+from tenacity import retry
 
 @pytest.fixture()
 def data_dir():
@@ -55,6 +56,7 @@ def extract_zip(cached_file, url, data_path):
     with zipfile.ZipFile(cached_file,"r") as zip_ref:
         zip_ref.extractall(data_path)
 
+@retry
 def download_url_to_file(url, dst, progress=True):
     r"""Download object at the given URL to a local path.
             Thanks to torch, slightly modified