internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: nilearn/nilearn

Issue:
Misleading (wrong?) code in Oasis VBM example
As pointed out on http://neurostars.org/p/3388/, the Oasis VBM code applies the NiftiMasker twice, and thus ends up smoothing the data twice.

It does that to do screening of low variance features. I think that the right way to do this would be to use a sklearn.feature_selection.VarianceThreshold in the pipeline.



Discussion:

Resolving patch:
diff --git a/examples/decoding/plot_oasis_vbm.py b/examples/decoding/plot_oasis_vbm.py
--- a/examples/decoding/plot_oasis_vbm.py
+++ b/examples/decoding/plot_oasis_vbm.py
@@ -63,12 +63,7 @@
     standardize=False,
     smoothing_fwhm=2,
     memory='nilearn_cache')  # cache options
-# remove features with too low between-subject variance
 gm_maps_masked = nifti_masker.fit_transform(gray_matter_map_filenames)
-gm_maps_masked[:, gm_maps_masked.var(0) < 0.01] = 0.
-# final masking
-new_images = nifti_masker.inverse_transform(gm_maps_masked)
-gm_maps_masked = nifti_masker.fit_transform(new_images)
 n_samples, n_features = gm_maps_masked.shape
 print("%d samples, %d features" % (n_subjects, n_features))
 
@@ -80,7 +75,11 @@
 svr = SVR(kernel='linear')
 
 # Dimension reduction
-from sklearn.feature_selection import SelectKBest, f_regression
+from sklearn.feature_selection import VarianceThreshold, SelectKBest, \
+        f_regression
+
+# Remove features with too low between-subject variance
+variance_threshold = VarianceThreshold(threshold=.01)
 
 # Here we use a classical univariate feature selection based on F-test,
 # namely Anova.
@@ -90,7 +89,10 @@
 # we can plug them together in a *pipeline* that performs the two operations
 # successively:
 from sklearn.pipeline import Pipeline
-anova_svr = Pipeline([('anova', feature_selection), ('svr', svr)])
+anova_svr = Pipeline([
+            ('variance_threshold', variance_threshold),
+            ('anova', feature_selection),
+            ('svr', svr)])
 
 ### Fit and predict
 anova_svr.fit(gm_maps_masked, age)
@@ -101,6 +103,8 @@
 coef = svr.coef_
 # reverse feature selection
 coef = feature_selection.inverse_transform(coef)
+# reverse variance threshold
+coef = variance_threshold.inverse_transform(coef)
 # reverse masking
 weight_img = nifti_masker.inverse_transform(coef)
 
@@ -137,13 +141,14 @@
 
 # Statistical inference
 from nilearn.mass_univariate import permuted_ols
+data = variance_threshold.fit_transform(gm_maps_masked)
 neg_log_pvals, t_scores_original_data, _ = permuted_ols(
-    age, gm_maps_masked,  # + intercept as a covariate by default
-    n_perm=1000,  # 1,000 in the interest of time; 10000 would be better
+    age, data,  # + intercept as a covariate by default
+    n_perm=2000,  # 1,000 in the interest of time; 10000 would be better
     n_jobs=1)  # can be changed to use more CPUs
 signed_neg_log_pvals = neg_log_pvals * np.sign(t_scores_original_data)
 signed_neg_log_pvals_unmasked = nifti_masker.inverse_transform(
-    signed_neg_log_pvals)
+    variance_threshold.inverse_transform(signed_neg_log_pvals))
 
 # Show results
 threshold = -np.log10(0.1)  # 10% corrected


Test patch:
Repository: nilearn/nilearn

Issue:
Globbing should expand user.
Globbing is good but we should also expand `~` to user home directory.



Discussion:
Cc @aabadie : do you want to do this during the sprint (and add it to the docs).

@GaelVaroquaux, I'll take it.

> @GaelVaroquaux, I'll take it.

No hurry, by the way. I don't think that we should try to squeeze this in
for the 0.2 release.


Resolving patch:
diff --git a/nilearn/_utils/niimg_conversions.py b/nilearn/_utils/niimg_conversions.py
--- a/nilearn/_utils/niimg_conversions.py
+++ b/nilearn/_utils/niimg_conversions.py
@@ -164,7 +164,9 @@ def check_niimg(niimg, ensure_ndim=None, atleast_4d=False, dtype=None,
     niimg: Niimg-like object
         See http://nilearn.github.io/building_blocks/manipulating_mr_images.html#niimg.
         If niimg is a string, consider it as a path to Nifti image and
-        call nibabel.load on it. If it is an object, check if get_data()
+        call nibabel.load on it. The '~' symbol is expanded to the user home
+        folder.
+        If it is an object, check if get_data()
         and get_affine() methods are present, raise TypeError otherwise.
 
     ensure_ndim: integer {3, 4}, optional
@@ -209,7 +211,8 @@ def check_niimg(niimg, ensure_ndim=None, atleast_4d=False, dtype=None,
 
     if isinstance(niimg, _basestring):
         if wildcards and ni.EXPAND_PATH_WILDCARDS:
-            filenames = sorted(glob.glob(niimg))  # Ascending sorting
+            # Ascending sorting + expand user path
+            filenames = sorted(glob.glob(os.path.expanduser(niimg)))
 
             # processing filenames matching globbing expression
             if len(filenames) >= 1 and glob.has_magic(niimg):


Test patch:
Repository: nilearn/nilearn

Issue:
Avoid documentation rot in continuous integration
I have just added a 'make check' target to the doc Makefile. It fails when there is a warning in the docs.

I have also fixed all the warnings (hours of work :( ).

I think that we should add "make check" as a test on circleci. It needs to be done after we build the docs, as the examples need to be run for warnings to pass.

@jnothman : we might be interested in porting "make check" to scikit-learn, and the CI test too, if the experiment in nilearn is successful. It's been such a fight with warnings in the scikit-learn docs.



Discussion:
I had a look at this one and I'm about to have something fully working on circle ci.

Here are my conclusions:
- make -C doc check requires packages to be installed on the Ubuntu VM (12.04): dvipng.
- installing extra packages was broken on the VM but I could fix the issue by removing the conflicting latex package (texlive-binaries).
- I'm now trying the identify the missing packages in order to have the full doc check running.

As I had problems with circle-ci at the beginning, I also tried to run the doc check on travis and noticed a few things:
- I couldn't succeed because of the output limit (4MB) being reached by the full test suite (including doc test/check).
- At the beginning I reached several times the maximum resources provided by the legacy infrastructure of travis-ci. I gave a try to the container based one using : `sudo: required` in the `.travis.yml` configuration file. It works. The container infrastructure should be faster (that's what travis-ci claims). Maybe this change can be part of another PR.


Resolving patch:
diff --git a/examples/connectivity/plot_canica_resting_state.py b/examples/connectivity/plot_canica_resting_state.py
--- a/examples/connectivity/plot_canica_resting_state.py
+++ b/examples/connectivity/plot_canica_resting_state.py
@@ -25,7 +25,7 @@
 # First we load the ADHD200 data
 from nilearn import datasets
 
-adhd_dataset = datasets.fetch_adhd()
+adhd_dataset = datasets.fetch_adhd(n_subjects=30)
 func_filenames = adhd_dataset.func  # list of 4D nifti files for each subject
 
 # print basic information on the dataset
diff --git a/examples/connectivity/plot_connectivity_measures.py b/examples/connectivity/plot_connectivity_measures.py
--- a/examples/connectivity/plot_connectivity_measures.py
+++ b/examples/connectivity/plot_connectivity_measures.py
@@ -13,7 +13,7 @@
 # Fetch dataset
 import nilearn.datasets
 atlas = nilearn.datasets.fetch_atlas_msdl()
-dataset = nilearn.datasets.fetch_adhd()
+dataset = nilearn.datasets.fetch_adhd(n_subjects=30)
 
 
 ######################################################################


Test patch:
Repository: nilearn/nilearn

Issue:
Cache directory in SpaceNet examples
In the examples :
- `examples/decoding/plot_mixed_gambles_space_net.py`
- `examples/decoding/plot_haxby_space_net.py`

`memory="nilearn_cache"` should be used rather than `memory="cache"`
As it is, I have two cache folders in `example/decoding`.  

ping @dohmatob 



Discussion:
OK, agreed there is some uniformaization to be done here. Thanks. Do you think you can do a PR for this ?

Yes


Resolving patch:
diff --git a/examples/decoding/plot_haxby_space_net.py b/examples/decoding/plot_haxby_space_net.py
--- a/examples/decoding/plot_haxby_space_net.py
+++ b/examples/decoding/plot_haxby_space_net.py
@@ -46,7 +46,7 @@
 from nilearn.decoding import SpaceNetClassifier
 
 # Fit model on train data and predict on test data
-decoder = SpaceNetClassifier(memory="cache", penalty='graph-net')
+decoder = SpaceNetClassifier(memory="nilearn_cache", penalty='graph-net')
 decoder.fit(X_train, y_train)
 y_pred = decoder.predict(X_test)
 accuracy = (y_pred == y_test).mean() * 100.
@@ -65,7 +65,7 @@
 
 ##############################################################################
 # Now Fit SpaceNet with a TV-l1 penalty
-decoder = SpaceNetClassifier(memory="cache", penalty='tv-l1')
+decoder = SpaceNetClassifier(memory="nilearn_cache", penalty='tv-l1')
 decoder.fit(X_train, y_train)
 y_pred = decoder.predict(X_test)
 accuracy = (y_pred == y_test).mean() * 100.
diff --git a/examples/decoding/plot_mixed_gambles_space_net.py b/examples/decoding/plot_mixed_gambles_space_net.py
--- a/examples/decoding/plot_mixed_gambles_space_net.py
+++ b/examples/decoding/plot_mixed_gambles_space_net.py
@@ -27,7 +27,7 @@
 from nilearn.decoding import SpaceNetRegressor
 decoder = SpaceNetRegressor(mask=mask_filename, penalty="tv-l1",
                             eps=1e-1,  # prefer large alphas
-                            memory="cache")
+                            memory="nilearn_cache")
 
 decoder.fit(zmap_filenames, behavioral_target)
 
@@ -41,7 +41,7 @@
 # Fit Graph-Net
 decoder = SpaceNetRegressor(mask=mask_filename, penalty="graph-net",
                             eps=1e-1,  # prefer large alphas
-                            memory="cache")
+                            memory="nilearn_cache")
 decoder.fit(zmap_filenames, behavioral_target)
 
 # Visualize Graph-Net weights
diff --git a/examples/decoding/plot_oasis_vbm_space_net.py b/examples/decoding/plot_oasis_vbm_space_net.py
--- a/examples/decoding/plot_oasis_vbm_space_net.py
+++ b/examples/decoding/plot_oasis_vbm_space_net.py
@@ -48,7 +48,7 @@
 #
 # Also, here we use a graph-net penalty but more beautiful results can be
 # obtained using the TV-l1 penalty, at the expense of longer runtimes.
-decoder = SpaceNetRegressor(memory="cache", penalty="graph-net",
+decoder = SpaceNetRegressor(memory="nilearn_cache", penalty="graph-net",
                             screening_percentile=5., memory_level=2)
 decoder.fit(gm_imgs_train, age_train)  # fit
 coef_img = decoder.coef_img_


Test patch:
Repository: nilearn/nilearn

Issue:
plot_stat_map(..., cut_coords=some_int, display_mode="ortho") throws criptic error
`plot_anat("funky.nii.gz", display_mode="ortho", cut_coords=6)`

```
TypeError: object of type 'int' has no len()
```

We should check when `cut_coords` is an int and `display_mode` is `"ortho"` and then take best possible action (with appropriate warning) or fail gracefully.



Discussion:
Not a great error message ! I would just go for a ValueError with a message like "with display_mode='ortho', 'cut_coords' needs to be a list of 3d world coordinates. You provided 'cut_coords=...' instead." or something better phrased.

> Not a great error message ! I would just go for a ValueError with a
> message like "with display_mode='ortho', 'cut_coords' needs to be a
> list of 3d world coordinates. You provided 'cut_coords=...' instead."
> or something better phrased.

+1 indeed. PR welcomed.

Anybody on this ? I can take care of this.

I don't think anybody is on it.


Resolving patch:
diff --git a/nilearn/plotting/img_plotting.py b/nilearn/plotting/img_plotting.py
--- a/nilearn/plotting/img_plotting.py
+++ b/nilearn/plotting/img_plotting.py
@@ -139,6 +139,11 @@ def _plot_img_with_bg(img, bg_img=None, cut_coords=None,
                    'Tip: Use np.nan_max() instead of np.max().')
         warnings.warn(nan_msg)
 
+    if isinstance(cut_coords, numbers.Number) and display_mode == 'ortho':
+        raise ValueError("The input given for display_mode='ortho' needs to be "
+                         "a list of 3d world coordinates in (x, y, z). "
+                         "You provided single cut, cut_coords={0}".format(cut_coords))
+
     if img is not False and img is not None:
         img = _utils.check_niimg_3d(img, dtype='auto')
         data = img.get_data()


Test patch:
diff --git a/nilearn/plotting/tests/test_img_plotting.py b/nilearn/plotting/tests/test_img_plotting.py
--- a/nilearn/plotting/tests/test_img_plotting.py
+++ b/nilearn/plotting/tests/test_img_plotting.py
@@ -88,6 +88,7 @@ def test_plot_anat():
     finally:
         os.remove(filename)
 
+
 def test_plot_functions():
     img = _generate_img()
 
@@ -787,3 +788,15 @@ def test_get_colorbar_and_data_ranges_masked_array():
     assert_equal(vmax, 2)
     assert_equal(cbar_vmin, None)
     assert_equal(cbar_vmax, None)
+
+
+def test_invalid_in_display_mode_cut_coords_all_plots():
+    img = _generate_img()
+
+    for plot_func in [plot_img, plot_anat, plot_roi, plot_epi,
+                      plot_stat_map, plot_prob_atlas, plot_glass_brain]:
+        assert_raises_regex(ValueError,
+                            "The input given for display_mode='ortho' needs to "
+                            "be a list of 3d world coordinates.",
+                            plot_func,
+                            img, display_mode='ortho', cut_coords=2)
Repository: nilearn/nilearn

Issue:
Discussion for a bug fix: view_type='filled_contours' bug in plot_prob_atlas plotting function
As per the latest release 1.5.0 matplotlib, taking zero in upper or lower level, simply ignores the masking values for colormap matplotlib contour fillings. As you can reproduce using this snippet:

``` python
from nilearn import datasets
from nilearn import plotting

atlas = datasets.fetch_atlas_msdl()
plotting.plot_prob_atlas(atlas.maps, view_type='filled_contours', display_mode='z', cut_coords=1)
```

![old_code](https://cloud.githubusercontent.com/assets/11410385/12151399/43b07c80-b4af-11e5-8a8b-3bfe758a73e7.png)

Here is a fix, I think:
Following discussions from https://github.com/matplotlib/matplotlib/issues/5477,
contour fillings levels should be given as (lower, upper) and what we need I believe in our case is that we force to append each upper level to `inf` to be well adapted to new code release with matplotlib. Correct me if I miss something in the discussion.

We can resume to normal mode by forcing upper level always to infinity at Line 543 in `displays.py`. 
`levels.append(0.)` to `levels.append(numpy.inf)` for this type of problem.
![new_code](https://cloud.githubusercontent.com/assets/11410385/12154059/9d6e3722-b4bd-11e5-9924-c161a3647d52.png)



Discussion:
ping @AlexandreAbraham

+1

More precision on this problem. In matplotlib <1.5, asking to color the levels [1., 0.] colors all values above one. It works because of a bug in matplotlib with decreasing values. The contour code has changed in 1.5 and now giving non increasing list of levels yields an unexpected behavior (the actual Picasso painting at the top of this issue). So the proposition of Kamalakar makes the code compliant with matplotlib and solves the problem for 1.5 without impacting the behavior in previous versions.

@KamalakerDadi if you a PR, it will be merged ASAP.

Yup, I am on it.

Awesome!


Resolving patch:
diff --git a/nilearn/plotting/displays.py b/nilearn/plotting/displays.py
--- a/nilearn/plotting/displays.py
+++ b/nilearn/plotting/displays.py
@@ -539,8 +539,8 @@ def add_contours(self, img, filled=False, **kwargs):
         if filled:
             colors = kwargs['colors']
             levels = kwargs['levels']
-            # Append lower boundary value to '0' for contour fillings
-            levels.append(0.)
+            # contour fillings levels should be given as (lower, upper).
+            levels.append(np.inf)
             alpha = kwargs['alpha']
             self._map_show(img, type='contourf', levels=levels, alpha=alpha,
                            colors=colors[:3])


Test patch:
Repository: nilearn/nilearn

Issue:
Change the default number of subjects in fetch_adhd from 40 to 30
The rational is to avoid blowing the cache of dowloads in circleCI.



Discussion:
Can we do this while keeping the same ratio ADHD/controls ?
Also, the sites are not ordered when loading ADHD dataset but I don't know if this was done on purpose ...

> Can we do this while keeping the same ratio ADHD/controls ?

We need to check. That's a good point!

CircleCI is downloading data over and over again. We should address this issue.

> We need to check. That's a good point!

@GaelVaroquaux with 30 subjects ratio is adhd/control 17/13

If we need to keep strict ordering, then we have to change the ordering according to the sites. Otherwise, I am ready with PR. Just that I changed `n_subjects = max_subjects` to `n_subjects = 30` and will change documentation accordingly.


Resolving patch:
diff --git a/nilearn/datasets/func.py b/nilearn/datasets/func.py
--- a/nilearn/datasets/func.py
+++ b/nilearn/datasets/func.py
@@ -391,15 +391,16 @@ def fetch_nyu_rest(n_subjects=None, sessions=[1], data_dir=None, resume=True,
                  session=session, description=fdescr)
 
 
-def fetch_adhd(n_subjects=None, data_dir=None, url=None, resume=True,
+def fetch_adhd(n_subjects=30, data_dir=None, url=None, resume=True,
                verbose=1):
     """Download and load the ADHD resting-state dataset.
 
     Parameters
     ----------
     n_subjects: int, optional
-        The number of subjects to load. If None is given, all the
-        40 subjects are used.
+        The number of subjects to load from maximum of 40 subjects.
+        By default, 30 subjects will be loaded. If None is given,
+        all 40 subjects will be loaded.
 
     data_dir: string, optional
         Path of the data directory. Used to force data storage in a specified
@@ -407,7 +408,7 @@ def fetch_adhd(n_subjects=None, data_dir=None, url=None, resume=True,
 
     url: string, optional
         Override download URL. Used for test only (or if you setup a mirror of
-        the data).
+        the data). Default: None
 
     Returns
     -------


Test patch:
Repository: nilearn/nilearn

Issue:
Decrease verbosity of download progress on CI
The logs of circleCI are very hard to work with, because the progress log of the downloaders drowns the information:
https://circleci.com/gh/nilearn/nilearn/1097

I am in favor of introducing an environment variable that decreases by something like 50 the number of times that the progress is printed. It would be checked in the datasets code, and set in the circleCI build scripts.



Discussion:
I'll have a look at this asap.

Thanks!


Resolving patch:
diff --git a/nilearn/datasets/utils.py b/nilearn/datasets/utils.py
--- a/nilearn/datasets/utils.py
+++ b/nilearn/datasets/utils.py
@@ -103,7 +103,7 @@ def _chunk_report_(cur_chunk_size, bytes_so_far, total_size, initial_size, t0):
         # Trailing whitespace is to erase extra char when message length
         # varies
         sys.stderr.write(
-            "\rDownloaded %d of %d bytes (%0.2f%%, %s remaining)"
+            "\rDownloaded %d of %d bytes (%.1f%%, %s remaining)"
             % (bytes_so_far, total_size, total_percent * 100,
                _format_time(time_remaining)))
 
@@ -153,13 +153,18 @@ def _chunk_read_(response, local_file, chunk_size=8192, report_hook=None,
         total_size = None
     bytes_so_far = initial_size
 
-    t0 = time.time()
+    t0 = time_last_display = time.time()
     while True:
         chunk = response.read(chunk_size)
         bytes_so_far += len(chunk)
-        if report_hook:
-            _chunk_report_(len(chunk), bytes_so_far, total_size, initial_size, t0)
-
+        time_last_read = time.time()
+        if (report_hook and
+                # Refresh report every half second or when download is
+                # finished.
+                (time_last_read > time_last_display + 0.5 or not chunk)):
+            _chunk_report_(len(chunk), bytes_so_far,
+                           total_size, initial_size, t0)
+            time_last_display = time_last_read
         if chunk:
             local_file.write(chunk)
         else:
@@ -531,7 +536,8 @@ def _fetch_file(url, data_dir, resume=True, overwrite=False,
         dt = time.time() - t0
         if verbose > 0:
             # Complete the reporting hook
-            sys.stderr.write(' ...done. (%i seconds, %i min)\n' % (dt, dt // 60))
+            sys.stderr.write(' ...done. ({0:.0f} seconds, {1:.0f} min)\n'
+                             .format(dt, dt // 60))
     except (_urllib.error.HTTPError, _urllib.error.URLError) as e:
         if 'Error while fetching' not in str(e):
             # For some odd reason, the error message gets doubled up


Test patch:
Repository: nilearn/nilearn

Issue:
Add a pointer to dev docs in the README.rst
We should add a pointer to the developer guidelines in the README.rst so that a new developer can get up to speed/



Discussion:
Sounds related to #891 ?


Resolving patch:
diff --git a/doc/conf.py b/doc/conf.py
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -26,13 +26,6 @@
 # We also add the directory just above to enable local imports of nilearn
 sys.path.insert(0, os.path.abspath('..'))
 
-try:
-    shutil.copy('../AUTHORS.rst', '.')
-except IOError:
-    # When nose scans this file, it is not in the right working
-    # directory, and thus the line above fails
-    pass
-
 # -- General configuration ---------------------------------------------------
 
 # Add any Sphinx extension module names here, as strings. They can be


Test patch: