Repository: nilearn/nilearn

Issue:
`fetch_atlas_aal` download error with `SPM5` and `SPM8`
With Nilearn 0.8.2.dev.
I haven't investigated further than trying all possible versions with cleaned data cache, so I might have missed something.
It seems to work for `SPM12` (the default), but fails on my machine with all other versions.

Code to reproduce:

```python
from nilearn import datasets
b = datasets.fetch_atlas_aal(version='SPM5')  # 'SPM8' gives same error
```
```
OSError                                   Traceback (most recent call last)
<ipython-input-97-6fd2a3e7006b> in <module>
----> 1 b = datasets.fetch_atlas_aal(version='SPM5')

~/GitRepos/nilearn-fork/nilearn/datasets/atlas.py in fetch_atlas_aal(version, data_dir, url, resume, verbose)
    938     see :footcite:`AAL_atlas`,
    939     and :footcite:`TZOURIOMAZOYER2002273`.
--> 940 
    941     Parameters
    942     ----------

~/GitRepos/nilearn-fork/nilearn/datasets/utils.py in _fetch_files(data_dir, files, resume, verbose, session)
    706         with requests.Session() as session:
    707             session.mount("ftp:", _NaiveFTPAdapter())
--> 708             return _fetch_files(
    709                 data_dir, files, resume=resume,
    710                 verbose=verbose, session=session)

~/GitRepos/nilearn-fork/nilearn/datasets/utils.py in _fetch_files(data_dir, files, resume, verbose, session)
    784             if os.path.exists(temp_dir):
    785                 shutil.rmtree(temp_dir)
--> 786             raise IOError('Fetching aborted: ' + abort)
    787         files_.append(target_file)
    788     # If needed, move files from temps directory to final directory.

OSError: Fetching aborted: Not a gzipped file (b'\x1f\x9d')
> /home/nicolas/GitRepos/nilearn-fork/nilearn/datasets/utils.py(786)_fetch_files()
    784             if os.path.exists(temp_dir):
    785                 shutil.rmtree(temp_dir)
--> 786             raise IOError('Fetching aborted: ' + abort)
    787         files_.append(target_file)
    788     # If needed, move files from temps directory to final directory.
```


Discussion:
I had a quick look. It seems like the issue happens at this line:

https://github.com/nilearn/nilearn/blob/e1ef7a3061a14bbb990f040fe64391885b5bca5d/nilearn/datasets/utils.py#L359

which is failing because gzip cannot read the archives for `SPM5` and `SPM8`. I tried downloading and opening the files manually and this is what I get:

```python
>>> import gzip

>>>  with gzip.open('aal_for_SPM5.tar.gz') as gz:
...:     gz.read()
...: 
BadGzipFile                               Traceback (most recent call last)
<ipython-input-8-e42e7d3e487c> in <module>
      1 with gzip.open('aal_for_SPM5.tar.gz') as gz:
----> 2     gz.read()
      3 
      4 

~/anaconda3/envs/nilearn/lib/python3.8/gzip.py in read(self, size)
    290             import errno
    291             raise OSError(errno.EBADF, "read() on write-only GzipFile object")
--> 292         return self._buffer.read(size)
    293 
    294     def read1(self, size=-1):

~/anaconda3/envs/nilearn/lib/python3.8/gzip.py in read(self, size)
    477                 # jump to the next member, if there is one.
    478                 self._init_read()
--> 479                 if not self._read_gzip_header():
    480                     self._size = self._pos
    481                     return b""

~/anaconda3/envs/nilearn/lib/python3.8/gzip.py in _read_gzip_header(self)
    425 
    426         if magic != b'\037\213':
--> 427             raise BadGzipFile('Not a gzipped file (%r)' % magic)
    428 
    429         (method, flag,

BadGzipFile: Not a gzipped file (b'\x1f\x9d')
```

Indeed, with `SPM5`:

```python
>>> with open('aal_for_SPM5.tar.gz', 'rb') as gz:
   ...:     print(gz.read(2))
   ...: 
   ...: 
b'\x1f\x9d'
```

With `SPM12`:

```python
>>> with open('aal_for_SPM12.tar.gz', 'rb') as gz:
   ...:     print(gz.read(2))
   ...: 
   ...: 
b'\x1f\x8b'
```
Does anybody know if `fetch_atlas_aal` ever worked with something else than the default `SPM12`?
The fecther doesn't seem to make any distinction between these versions besides the download URLs. But, when manually downloading the data, the `atlas` folder is missing for versions 5 and 8:

```
$ wget http://www.gin.cnrs.fr/AAL_files/aal_for_SPM12.tar.gz
$ tar -xvzf aal_for_SPM12.tar.gz
aal/
aal/gin_list_dlabels.m
aal/gin_clusters.m
aal/atlas/
aal/atlas/AAL.nii
aal/atlas/AAL.xml
aal/gin_rclusters.m
aal/gin_det_plabels.m
aal/ROI_MNI_V4.txt
aal/ROI_MNI_V4.xml
aal/gin_list_plabels.m
aal/gin_det_dlabels.m
aal/aal.m
aal/ROI_MNI_V4.nii
aal/ROI_MNI_V4_Border.mat
aal/gin_clusters_plabels.m
aal/gin_dlabels.m
aal/readme_aal_for_SPM12.txt
aal/ROI_MNI_V4_List.mat
```

```
$ wget http://www.gin.cnrs.fr/AAL_files/aal_for_SPM5.tar.gz
$ tar -xvzf aal_for_SPM5.tar.gz
aal/
aal/aal.m
aal/ROI_MNI_V4.txt
aal/gin_det_dlabels.m
aal/gin_det_plabels.m
aal/gin_clusters.m
aal/gin_dlabels.m
aal/ROI_MNI_V4_List.mat
aal/ROI_MNI_V4_Border.mat
aal/gin_list_dlabels.m
aal/gin_list_plabels.m
aal/gin_clusters_plabels.m
aal/gin_rclusters.m
aal/gin_list_dlabels.m~
aal/gin_list_plabels.m~
aal/readme_aal5.txt
aal/ROI_MNI_V4.nii
```

which will be an issue at these lines:

https://github.com/nilearn/nilearn/blob/e1ef7a3061a14bbb990f040fe64391885b5bca5d/nilearn/datasets/atlas.py#L906-L913
I don't know, sorry! so besides the file format issue, the downloaded archives for 5 and 8 do not actually contain the atlas?
>so besides the file format issue, the downloaded archives for 5 and 8 do not actually contain the atlas?

This is what I understand too. This is why I am curious to know if it ever worked. It looks like these options were added a while ago in #802, but I couldn't find any issue related to spm 5 and 8... :thinking: 
maybe they are so old no one uses them?


Resolving patch:
diff --git a/nilearn/datasets/atlas.py b/nilearn/datasets/atlas.py
--- a/nilearn/datasets/atlas.py
+++ b/nilearn/datasets/atlas.py
@@ -1093,32 +1093,43 @@ def fetch_atlas_aal(version='SPM12', data_dir=None, url=None, resume=True,
                          'Please choose one among %s.' %
                          (version, str(versions)))
 
-    if url is None:
-        baseurl = "http://www.gin.cnrs.fr/AAL_files/aal_for_%s.tar.gz"
-        url = baseurl % version
-    opts = {'uncompress': True}
-
     dataset_name = "aal_" + version
-    # keys and basenames would need to be handled for each spm_version
-    # for now spm_version 12 is hardcoded.
-    basenames = ("AAL.nii", "AAL.xml")
-    filenames = [(os.path.join('aal', 'atlas', f), url, opts)
-                 for f in basenames]
+    opts = {'uncompress': True}
 
-    data_dir = _get_dataset_dir(dataset_name, data_dir=data_dir,
-                                verbose=verbose)
-    atlas_img, labels_file = _fetch_files(data_dir, filenames, resume=resume,
-                                          verbose=verbose)
+    if url is None:
+        if version == 'SPM12':
+            url = "http://www.gin.cnrs.fr/AAL_files/aal_for_SPM12.tar.gz"
+            basenames = ("AAL.nii", "AAL.xml")
+            filenames = [(os.path.join('aal', 'atlas', f), url, opts)
+                         for f in basenames]
+        else:
+            url = f"http://www.gin.cnrs.fr/wp-content/uploads/aal_for_{version}.zip"  # noqa
+            basenames = ("ROI_MNI_V4.nii", "ROI_MNI_V4.txt")
+            filenames = [(os.path.join(f'aal_for_{version}', f), url, opts)
+                         for f in basenames]
 
-    fdescr = _get_dataset_descr(dataset_name)
-    # We return the labels contained in the xml file as a dictionary
-    xml_tree = xml.etree.ElementTree.parse(labels_file)
-    root = xml_tree.getroot()
+    data_dir = _get_dataset_dir(
+        dataset_name, data_dir=data_dir, verbose=verbose
+    )
+    atlas_img, labels_file = _fetch_files(
+        data_dir, filenames, resume=resume, verbose=verbose
+    )
+    fdescr = _get_dataset_descr("aal_SPM12")
     labels = []
     indices = []
-    for label in root.iter('label'):
-        indices.append(label.find('index').text)
-        labels.append(label.find('name').text)
+    if version == 'SPM12':
+        xml_tree = xml.etree.ElementTree.parse(labels_file)
+        root = xml_tree.getroot()
+        for label in root.iter('label'):
+            indices.append(label.find('index').text)
+            labels.append(label.find('name').text)
+    else:
+        with open(labels_file, "r") as fp:
+            for line in fp.readlines():
+                _, label, index = line.strip().split('\t')
+                indices.append(index)
+                labels.append(label)
+        fdescr = fdescr.replace("SPM 12", version)
 
     params = {'description': fdescr, 'maps': atlas_img,
               'labels': labels, 'indices': indices}


Test patch:
diff --git a/nilearn/datasets/tests/test_atlas.py b/nilearn/datasets/tests/test_atlas.py
--- a/nilearn/datasets/tests/test_atlas.py
+++ b/nilearn/datasets/tests/test_atlas.py
@@ -435,28 +435,49 @@ def test_fetch_atlas_difumo(tmp_path, request_mocker):
                                  dimension=128, resolution_mm=3.14)
 
 
-def test_fetch_atlas_aal(tmp_path, request_mocker):
-    metadata = (b"<?xml version='1.0' encoding='us-ascii'?>"
-                b"<metadata></metadata>")
-    archive_root = Path("aal", "atlas")
+@pytest.fixture
+def aal_archive_root(version):
+    if version == 'SPM12':
+        return Path("aal", "atlas")
+    else:
+        return Path(f"aal_for_{version}")
+
+
+@pytest.mark.parametrize("version,archive_format,url_key",
+                         [('SPM5', 'zip', 'uploads'),
+                          ('SPM8', 'zip', 'uploads'),
+                          ('SPM12', 'gztar', 'AAL_files')])
+def test_fetch_atlas_aal(version, archive_format, url_key, aal_archive_root,
+                         tmp_path, request_mocker):
+    metadata = "A\tB\tC\n"
+    if version == 'SPM12':
+        metadata = (b"<?xml version='1.0' encoding='us-ascii'?>"
+                    b"<metadata><label><index>1</index>"
+                    b"<name>A</name></label></metadata>")
+    label_file = "AAL.xml" if version == 'SPM12' else "ROI_MNI_V4.txt"
+    atlas_file = "AAL.nii" if version == 'SPM12' else "ROI_MNI_V4.nii"
     aal_data = dict_to_archive(
-        {archive_root / "AAL.xml": metadata, archive_root / "AAL.nii": ""})
-
-    request_mocker.url_mapping["*AAL_files*"] = aal_data
-    dataset = atlas.fetch_atlas_aal(data_dir=tmp_path, verbose=0)
+        {aal_archive_root / label_file: metadata,
+         aal_archive_root / atlas_file: ""},
+        archive_format=archive_format
+    )
+    request_mocker.url_mapping[f"*{url_key}*"] = aal_data
+    dataset = atlas.fetch_atlas_aal(
+        version=version, data_dir=tmp_path, verbose=0
+    )
     assert isinstance(dataset.maps, str)
     assert isinstance(dataset.labels, list)
     assert isinstance(dataset.indices, list)
     assert request_mocker.url_count == 1
+    assert dataset.description != ''
 
-    with pytest.raises(ValueError,
-                       match='The version of AAL requested "FLS33"'
-                       ):
-        atlas.fetch_atlas_aal(version="FLS33",
-                              data_dir=tmp_path,
-                              verbose=0)
 
-    assert dataset.description != ''
+def test_fetch_atlas_aal_version_error(tmp_path, request_mocker):
+    with pytest.raises(ValueError,
+                       match='The version of AAL requested "FLS33"'):
+        atlas.fetch_atlas_aal(
+            version="FLS33", data_dir=tmp_path, verbose=0
+        )
 
 
 def test_fetch_atlas_basc_multiscale_2015(tmp_path, request_mocker):