Repository: nilearn/nilearn

Issue:
Datasets file fetcher does not respect `requests.Session` instance
When providing a `requests.Session` instance to the `session` argument of `nilearn.datasets.utils._fetch_files()` the session instance is not passed through to the calls to `nilearn.datasets.utils._fetch_file()` so any e.g., header information that should be sent with the requests is lost. While these functions accept `username` and `password` arguments for basic authentication, it's unclear how to provide e.g., token-based authentication without the ability to directly pass a session instance.

<!--Please fill in the following information, to the best of your ability.-->
Nilearn version: 0.7.1

### Expected behavior

Passing a `requests.Session` instance to the `session` keyword of `nilearn.datasets.utils._fetch_files()` should cause any header information stored in the session instance to be used when fetching the requested files.

### Actual behavior

The `session` argument of `nilearn.datasets.utils._fetch_files()` is entirely ignored.

### Steps and code to reproduce bug

Since the issue I'm running into is in providing token-based authentication via the session header this isn't a functional MWE but should hopefully highlight the issue:

```python
>>> import requests
>>> from nilearn.datasets.utils import _fetch_files
>>> session = requests.Session()
>>> session.headers['Authentication'] = 'Bearer TOKEN'
>>> _fetch_files('/tmp', files=('filename.txt', 'https://authenticationrequired.com/filename.txt', {}), session=session) 
```

The last line will raise the error:

```python-traceback
~/lib/python3.8/site-packages/nilearn/datasets/utils.py in _fetch_files(data_dir, files, resume, verbose, session)
    776             md5sum = opts.get('md5sum', None)
    777 
--> 778             dl_file = _fetch_file(url, temp_dir, resume=resume,
    779                                   verbose=verbose, md5sum=md5sum,
    780                                   username=opts.get('username', None),

~/lib/python3.8/site-packages/nilearn/datasets/utils.py in _fetch_file(url, data_dir, resume, overwrite, md5sum, username, password, verbose, session)
    614             with session.send(
    615                     prepped, stream=True, timeout=_REQUESTS_TIMEOUT) as resp:
--> 616                 resp.raise_for_status()
    617                 with open(temp_full_name, "wb") as fh:
    618                     _chunk_read_(resp, fh, report_hook=(verbose > 0),

~/lib/python3.8/site-packages/requests/models.py in raise_for_status(self)
    941 
    942         if http_error_msg:
--> 943             raise HTTPError(http_error_msg, response=self)
    944 
    945     def close(self):

HTTPError: 401 Client Error: Unauthorized for url: https://authenticationrequired.com/filename.txt
```

### Potential fix

Adding `session=session` to the following call in `nilearn.datasets.utils._fetch_files()` should remedy the issue:

https://github.com/nilearn/nilearn/blob/dbbd221e3633c97464034c5366faa548728c9999/nilearn/datasets/utils.py#L778-L782

I recognize this isn't a user-facing function but given the keyword parameter is in the function signature and you [make a point of re-calling the function with a `requests.Session` instance if one isn't provided](https://github.com/nilearn/nilearn/blob/dbbd221e3633c97464034c5366faa548728c9999/nilearn/datasets/utils.py#L726-L731) makes it seem like it should be respected.

---

Let me know if you have any questions about this and, as always, thanks for the great package! :sparkles: 


Discussion:
thanks for reporting! indeed it looks like an omission; I can open a PR.

(as a side note, as these functions are not part of the public api, and are burdened with a fair amount of historical cruft, I really don't recommend using them directly at all)

Resolving patch:
diff --git a/nilearn/datasets/utils.py b/nilearn/datasets/utils.py
--- a/nilearn/datasets/utils.py
+++ b/nilearn/datasets/utils.py
@@ -779,7 +779,7 @@ def _fetch_files(data_dir, files, resume=True, verbose=1, session=None):
                                   verbose=verbose, md5sum=md5sum,
                                   username=opts.get('username', None),
                                   password=opts.get('password', None),
-                                  overwrite=overwrite)
+                                  session=session, overwrite=overwrite)
             if 'move' in opts:
                 # XXX: here, move is supposed to be a dir, it can be a name
                 move = os.path.join(temp_dir, opts['move'])


Test patch:
diff --git a/nilearn/datasets/tests/test_utils.py b/nilearn/datasets/tests/test_utils.py
--- a/nilearn/datasets/tests/test_utils.py
+++ b/nilearn/datasets/tests/test_utils.py
@@ -12,7 +12,7 @@
 import zipfile
 import urllib
 import json
-
+from unittest.mock import MagicMock
 from tempfile import mkdtemp, mkstemp
 
 try:
@@ -313,6 +313,20 @@ def test_fetch_file_overwrite(tmp_path, request_mocker):
         assert fp.read() == ''
 
 
+def test_fetch_files_use_session(tmp_path, request_mocker):
+    # regression test for https://github.com/nilearn/nilearn/issues/2863
+    session = MagicMock()
+    datasets.utils._fetch_files(
+        files=[
+            ("example1", "https://example.org/example1", {"overwrite": True}),
+            ("example2", "https://example.org/example2", {"overwrite": True}),
+        ],
+        data_dir=str(tmp_path),
+        session=session,
+    )
+    assert session.send.call_count == 2
+
+
 def test_fetch_files_overwrite(tmp_path, request_mocker):
     # overwrite non-exiting file.
     files = ('1.txt', 'http://foo/1.txt')