Repository: nilearn/nilearn

Issue:
Automate the Formatting stuff
Prompted by https://github.com/nilearn/nilearn/pull/2477#issuecomment-648905550

We have occasionally discussed this, but I think now is the time.

Nilearn contributors, managers, core developers juggle a lot of other responsibilities. We should automate the stuff that we can.

There are automated formatters & linters as well as cloud based code quality platforms that would automate a lot of minutia, making it a non-issue for contributors & reviewers. We can set up our custom for these tools & then we should start incorporating those.


Discussion:
I can imagine this subject is often being discussed...
I personally enjoyed contributing to repos which implemented some `pre-commit` logic with `black` for instance (I know it has pros and cons but at least it was very easy to setup and quite straight forward).
> I can imagine this subject is often being discussed...
> I personally enjoyed contributing to repos which implemented some `pre-commit` logic with `black` for instance (I know it has pros and cons but at least it was very easy to setup and quite straight forward).

We want to make it as easy as possible to start contributing even for
people who aren't very comfortable with git, so asking everyone to
install pre-commit hooks seems a bit too much

Locally run pre-commit hooks are often a big stumble. I would not recommend enforcing them.
I was thinking about running in-cloud linters and formatters, such as : https://github.com/marketplace/actions/run-black-formatter

There are a bunch on github marketplace https://github.com/marketplace?query=formatter
@jeromedockes, I get your point, but couldn't we write the `pre-commit` config file and simply ask the user to [download `pre-commit`](https://pre-commit.com/#installation)?
Actually, it's nothing more than a `pip install pre-commit`, we could even include this package in `requirements-dev.txt`.

The tricky part maybe is when the user has to `black` (or any other code-formatter) their code, but again, the `pre-commit` logic can take care of this.

So my overall feeling is that, I done well, it actually wouldn't add any work on the contributor's end (except run `git commit` once more after the `pre-commit` logic triggered by the first `git commit ` formatted their code). 

@kchawla-pi, I think this is a nice idea too: maybe we could lint code when we merge PRs, so that all code present on `master` is formatted?
No. The pre-commit hooks don't work well if a user uses an IDE which does not link pre-commit hooks to a custom python such as conda's or a virtual environment. 

For example I have Conda's Python set as my default in Linux and I always use virtual environments and unless I install the pre-commit hook in the system python3 Pycharm always fails the commit process.

But the thing is it took me a while to figure all this out and until I did it was a big hassle to commit.

We cannot afford this kind of friction. 
I also don't like the idea of a repo I am contributing to dictates how I am setting up my system's python environment.

I managed to fix this in Pycharm by finally making the necessary changes I mentioned above. But that was on my work laptop, I was literally being paid to do it. SO I swallowed my preferences and changes how my system was setup. But we must not do this to our contributors.
> I think this is a nice idea too: maybe we could lint code when we merge PRs, so that all code present on master is formatted?

I think that is a better approach. But its for @jeromedockes  and @emdupre  to decide.
Edit: And @NicolasGensollen to decide. Oops. :-)
@NicolasGensollen hey i just came across this. Haven't looked at it yet but could be useful https://deepsource.io/python-static-code-analysis/
@kchawla-pi @alexisthual I will put this on the list of topics for the next core-devs meeting.

In addition to the increase of complexity for users mentioned by @jeromedockes, I am a bit worried by the transition phase. 
I have some bad memories of another project where we decided to use black at some point. This was causing huge diffs in PRs because of the reformatting. Stalled PRs became almost impossible to review and so on...
The code base was much smaller than nilearn's and it was a pain. (Note that there might be more clever way to do this transition...).

>hey i just came across this. Haven't looked at it yet but could be useful

Thanks for the link @kchawla-pi ! 
I never used it so I will have to take a look first to understand what it does and if it would be helpful for Nilearn.
The linter is currently configured to only checks changed lines, right? Could we add `flake8-black` and `flake8-isort` to the test dependencies to enforce their code style, but only on changed lines in each PR?
Hi everyone.

Since this issue was discussed, the software components that were an issue have been refined and fixed. I have implemented pre-commit hooks in two commercial projects without any problem, and they are performing as expected, automatically linting and fixing changed code, and flake8 indicating the problems and blocking commits until resolved..

Once setup in the repo, a contributor can activate them before first use.
They can also be switched off temporarily or permanently if a contributor so desires.

I have a few days available to work on this. If the Nilearn team is amenable, I would be happy to implement this.

Let me know. 

Hi @kchawla-pi 
Sounds great. It would be awesome indeed if you can make our framework more up-to-date !
Best,
but regardless of whether the tools have improved, the issues pointed out in this issue and in #3317 and #3308 (one more thing to ask of contributors, one more thing to maintain) are still relevant. are these projects public, so that we can have a quick look at what this would look like?
To reiterate some old discussion. They are indeed super useful for devs who are familiar with the general development workflow, but there's much more variability in nilearn contributors, especially first timer. As much I enjoy working on projects with pre-commit, I am still skeptical about the cost and benefits for community maintained research code.
For @jeromedockes and those who are interested - Pydra has implemented precommit in their [task template](https://github.com/nipype/pydra-tasks-template)
It will be great if we can have a look at other projects with precommit implemented.

Will add my 2 cents as a pre-commit user and to mention one thing that I have not seen discussed.

Using pre-commit CI to run pre-commit on PR: https://pre-commit.ci/

### PROs
- no need to set up extra CI workflows:
  - pushes reformatting automatically on a PR where possible (for example. black): see [example](https://github.com/cpp-lln-lab/bidsMReye/commit/d0bf5d76cad2a04b28fbbcd7429b80c65c7ab517) (definitely more aggressive than sourcery [opening PRs on your PR to suggest refactoring](https://github.com/cpp-lln-lab/bidsMReye/pull/41))
  - shows as failed CI check when reformatting is not in question (for exmaple. flake8)
  - shows up as green when everthing is fine ([example](https://results.pre-commit.ci/run/github/208231636/1659186827.qrw7gw94Sg-G4IaxY8Dlyw))
- no need for local set up to use pre-commit (reduce friction for first timers or forcing more advanced users to use a special set up)
- auto-updates the pre-commit hooks to their latest version: [example](https://github.com/cpp-lln-lab/CPP_SPM/pull/732)

### CONs
- adds extra commit that can confuse first timers (after pushing on a PR one may have to pull the pre-commit-ci changes before   being able to push again)
- worst case of the thing above can also create merge conflicts even for one user working alone on the same branch which can be VERY confusing (never happened in my experience)

Hope this helps making an informed choice that helps both maintainers and contributors.
Thx for the comments. Indeed we should be worried about non-expert users, who often have a hard time with the pull/push operations and their ordering.
Thanks all for the discussion. I think indeed if we do decide to implement any automation we really need to think about how it will affect non-expert contributors. Another point that was brought up in previous discussions is that it would be preferable to do it in the way that linting and formatting would only be done on changed lines as to not cause huge diffs.

@kchawla-pi I'm curious about your suggestion, can you give a bit more detail to address some of the concerns raised here? Better yet if there is an example project you can provide.

@Remi-Gau thanks for the suggestion, I will have to look more into it as I'm not really familiar with this CI.
Pretty sure that pre commit ci will work on entire files (as would pre commit locally) so I can defo imagine the potential for merge conflicts nightmares if you throw black in there. 

For a large code base with many open pr doing a `pre-commit run - a` would be dangerous. If you went down that route it would have to be applied gradually and first on the most "stable" files. Could require some planning and extra efforts that may pay off in the long run but they may not...
So I understand the reluctance.

OK I will shut up now. 
Hi all. What a rousing discussion! :~) 

I will address the concerns raised.

Burden for maintainers
-----------------------
The pre-commit hooks will not be setup manually, but by using the pre-commit python library which makes this a very simple and straightforward undertaking. It will only add a developmental dependency, not a usage dependency.  
Setting up only requires configuring a file, choosing which tools (hooks) to run, their version, and URL to download the tool from. The download is done only once on the local device, at setup time.
After initial setup, it could be 2-3 years before a version needs to be updated, if at all. Usually this is done only to fix linting rule conflicts or more functionality. If ignored, typically there is no further maintenance required for the pre-commit library and hooks.

The only real effort will be the first definition of rules, tailoring the to Nilearn's preferences, and making sure there are no conflicts among them. Which I will do.

The pre-commit package and the hooks are all open source, most developed by well regarded organisations, like Python Software Foundation, AIrBnB, Netflix etc.

Burden for Contributors
------------------------
The extra burden to us pre-commit hooks for contributors will be that they will have to activate the pre-commit hooks once on their local system. To do this they will have to run the command `pre-commit install` in the Nilearn root directory. That's it.
If they do not run this command, the pre-commit hooks will not activate, and their workflow will remain completely unchanged.  
If contributors want to, at any point of time, running `pre-commit uninstall`, or unchecking the option in their IDE will remove the hooks.
If contributors want to skip running them for a specific commit, they can uncheck the option for that specific commit in their IDE or do `git commit --skip-precommit`.

As for the errors pre-commit hooks will raise for not adhering to the rules, such as flake8, contributors are already expected to adhere to them and these are enforced during code reviews. 

Updating the contribution guide with what to expect with pre-commit hooks, screenshots of what the raised erros might look like and how to fix them, will take care of much of the friction. Which I shall do.

Ease for Maintainers
---------------------
Instead of doing code quality enforcement manually by reviewers via code review comments, it will be done automatically, and maintainers won't have to spend their resources over this.

Ease for Contributors
----------------------
Instead of having their PRs stalled, sometimes for days over formatting woes - indeed, that is what started this whole thread, contributors can focus on more substantive feedback. It will also slowly teach them better coding practices.

Concern over large number of changes and merge conflicts
------------------------------------------------------------
After doing this PR, I can do one more PR, this one only containing code linting and reformatting, not functional changes, and so essentially, nothing to review.  Subsequently, new PRs will not have huge reformatting.
Git is smart enough to not create merge conflicts due to formatting changes.  

Other Concerns
----------------
Black is now smart enough to not mess up single and double nested quotes.  

Finally, instead of running the hooks on each commit, which is the default behaviour, we can set it up so it is only run in the CI and only pre-merge - a pre-merge hook instead of a pre-commit hook. This will remove the contributor's participation in the process and the maintainer can pull the PR locally, run the formatting and linting, push it back and merge it.  
I do not think the contributors will be so enburdened by this that we need to do this, but it is an option.


Why We Should Do This Regardless of Any Objections
------------------------------------------------------
Reviewers already spend a lot of time doing this themselves or manually directing contributors. In every PR a lot of energy is directed towards these already, and we explain to new contributors what to do, and why to do it anyway. It is not like the new contributors don't have to learn all of that anyway, so this will not suddenly be an overwhelming burden of grief on them.
As the Nilearn project grows, maintaining code consistency will become even more important, and automating the process, shortening the feedback cycle, will simplify the lives of maintainers and contributors.

I implemented pre-commit hooks, and explained their purpose and functioning to two separate teams of 5, all data scientists and no software engineers, who were building a commercial product, and they all grasped it quickly. I don't believe Nilearn contributors will need extra help and guidance from maintainers beyond what already must be provided to new contributors.

For example, we explain to first time open-source contributors about how to write tests, how to use appropriate data and test cases, and how to use Pytest effectively. This should be trivial compared to all of that. 

@ymzayek I would love to provide and example but my implementations were all for closed sourced repos. I will build one for my project, or for Nilearn, and you all don't have to merge it if you don't like it.
Developer Experience is a very appropriate tag
Thanks for this nice summary, @kchawla-pi! I think it would be better if we
reached a decision (at least in principle) before you do the actual work of
writing the configuration files and opening the PR, to avoid wasting your time
if we eventually decide not to automate formatting.
To reiterate, even if this is implemented and merged, it needs to be turned on for first use by the contributor on their local system and can be turned off.

So in the contribution guide, it can be offered as an option for contributors who are happy to use it, and those who dislike it can ignore it or switch it off. 


also could you list the considered hooks and what they do? the discussion mentions code quality and linting so this is not just about formatting code with black any more?
https://pre-commit.com/hooks.html

To start with:

- isort
- black
- flake8
- bandit
- pydocstyle (maybe)
btw the discussion linked in this issue involved bugs, a function's signature,
and the quality of unit tests so the PR's delay was not due to formatting but
more fundamental issues. More generally a lot of the formatting stuff is
(almost) irrelevant. If we want to avoid asking too much of contributors, the
time they are willing to give (and the reviewers') is better spent improving the
design and implementation of the contribution, the correctness and modularity of
the code, defining clear interfaces, choosing meaningful names for types,
functions and variables, stepping through the code in a debugger, writing
effective tests, polishing the docstrings' content, etc. rather than fighting
merge conflicts introduced by a tool that changed single quotes to double
quotes, added commas at the ends of parameter lists or sorted imports. I do like
automatic formatting in general and I do use it, I just think it should be 
low on our priority list of things we ask for in a contribution and CI
configurations we are willing to maintain.
> To start with:

thanks! In the link I also see pylint; we may want to add that to the list of candidates, I find it often has helpful warnings
I was referring to the "rant" in the comment by dangom, not the whole PR.

Yes we can and do all the above things but we also do manual formatting and linting. Also we have always maintained flake8 checks, which include formatting etc.

Anyway I don't have any extra skin in the game. I will post an example, and you all can let me know.
I didn't understand this comment though https://github.com/nilearn/nilearn/issues/2528#issuecomment-1205653116

The current PR proposal does not interfere with any of the tasks mentioned in the comment linked above. It is simply automating something that is currently done manually. 

The relevance of this is the same as that of PEP8 that we have rigorously enforced all these years. This is simply automating the enforcement.

Beyond just formatting and linting, there are tools which can red flag code based on its complexity and quality. We can also look into that at a later date. BUt they are run in the CI, not part of the pre-commit proposal here.
I just meant we shouldn't overestimate the importance of formatting. But you are
right, we do strive to enforce pep8. although note that black cannot enforce
some of the most important aspects of pep8, which are not about formatting (eg
naming conventions, distinction of public, subclass and internal interfaces,
putting only what is needed in a try clause, catching specific exceptions,
correctly chaining exceptions, ...), so these parts still need to be addressed
manually. (conversely, black can modify code that complies with pep8.)

In any case, these were just some a priori concerns, but if the suggested
configuration does make new (possibly a bit unfamiliar with git) contributors'
and maintainers' life easier then it is great!
@kchawla-pi thanks for the thoroughness! I would be open to trying this. I think we need to weigh the pros and cons and make a decision (I agree with @jeromedockes comment https://github.com/nilearn/nilearn/issues/2528#issuecomment-1205611153 so as to not waste your time). 

@jeromedockes perhaps automatic formatting, even if it's not the most important aspect of pep8, would give us more time to focus on those other aspects you listed.

We can prioritize this at our next developers' meeting. But that won't be until mid-end September. 

Hehe, that's what Gensollen said last time! https://github.com/nilearn/nilearn/issues/2528#issuecomment-836532130
> @kchawla-pi @alexisthual I will put this on the list of topics for the next core-devs meeting.

There are a bunch of approved PRs that can be merged. If we merge those, the merge conflict concern will be obviated.
maybe we could talk about it in the next office hours rather than dev meeting so that @kchawla-pi  can participate more easily?

> There are a bunch of approved PRs that can be merged. If we merge those, the merge conflict concern will be obviated.

there would still be all the PRs that are not ready to be merged, and there will always be quite a few
Sure, I remember what it is like. But I noticed many are at least a year
old, and cannot reasonably be protected from merge conflicts at this point.



On Fri, Aug 5, 2022, 19:12 Jérôme Dockès ***@***.***> wrote:

> maybe we could talk about it in the next office hours rather than dev
> meeting so that @kchawla-pi <https://github.com/kchawla-pi> can
> participate more easily?
>
> There are a bunch of approved PRs that can be merged. If we merge those,
> the merge conflict concern will be obviated.
>
> there would still be all the PRs that are not ready to be merged, and
> there will always be quite a few
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/nilearn/nilearn/issues/2528#issuecomment-1206671986>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB6SXRHF3NNDHTVXEGP54UDVXVDQTANCNFSM4OHMSDXA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

> maybe we could talk about it in the next office hours rather than dev meeting so that @kchawla-pi can participate more easily?

Of course, that's ok with me!
> maybe we could talk about it in the next office hours rather than dev meeting so that @kchawla-pi can participate more easily?

Wait what? I will have to talk to *Jerome*??? I rescind this issue. We don't need any pre-commit hooks.
Tell me when we will talk. 
@jeromedockes and @kchawla-pi have you already had a chance to discuss this? If not, may I suggest meeting during the nilearn office hours? Would this Friday or the next (30th) work? If not, let me know some other options.
No, we haven't. This Friday would work; I think we were planning to talk
about surface data on the 30th but in any case I will be there as well.

Oh yes that's right about the 30th. Let's say this Friday if @kchawla-pi is able, otherwise we can pick a Friday in October.
What are the office hour times and which zone?

On Wed, Sep 21, 2022, 14:45 Yasmin ***@***.***> wrote:

> Oh yes that's right about the 30th. Let's say this Friday if @kchawla-pi
> <https://github.com/kchawla-pi> is able, otherwise we can pick a Friday
> in October.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/nilearn/nilearn/issues/2528#issuecomment-1253657953>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB6SXRC6NR4S6XYWQSH33RTV7L7PPANCNFSM4OHMSDXA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

@kchawla-pi it's held every Friday at 4pm to 5pm UTC. This link shows your local time and the link to join the Discord server: https://arewemeetingyet.com/UTC/2022-05-13/16:00/w/Nilearn%20Office%20Hours#eyJ1cmwiOiJodHRwczovL2Rpc2NvcmQuZ2cvYk1CaGI3dyJ9
I'm not available this Friday.


On Wed, Sep 21, 2022, 15:02 Yasmin ***@***.***> wrote:

> @kchawla-pi <https://github.com/kchawla-pi> it's held every Friday at 4pm
> to 5pm UTC. This link shows your local time and the link to join the
> Discord server:
> https://arewemeetingyet.com/UTC/2022-05-13/16:00/w/Nilearn%20Office%20Hours#eyJ1cmwiOiJodHRwczovL2Rpc2NvcmQuZ2cvYk1CaGI3dyJ9
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/nilearn/nilearn/issues/2528#issuecomment-1253679726>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB6SXRCOHXPNG26WYO63HZDV7MBPBANCNFSM4OHMSDXA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

What about in October?
I am available this Friday. Works for you?
For me it works though I might have to leave a bit early. @jeromedockes will you be there as well?
> @jeromedockes will you be there as well?

yes I'll be there!

Hey @ymzayek and @jeromedockes I am very sorry, I did not receive a notification from Github that you two have responded.
Would you be amenable to doing it on Friday this week?
I will ensure I keep an eye out for updates to this thread.
@kchawla-pi no worries, I realized I didn't ping you. For me this Friday works well too.
Thanks @ymzayek I will see you on the discord link you posted earlier on Friday, Nov 4 at 1200h my time and 1600h UTC.
Hey I am at the server but cant message there for 10 minutes as a new member

@ymzayek @jeromedockes 
Jerome an I are there.

Can you join the "general" voice channel?

In 6 more minutes
I can read the general voice channel but ont post for another 6 minutes

@kchawla-pi I am wondering if we could pick this back up again? With the flake8 breaking change removing `--diff` we cannot automate the PEP8 check on the diff anymore so it seems more worth the effort to do this now. Also I saw a strategy outlined by [astropy/astropy](https://github.com/astropy/astropy) for implementing black to reformat their library which might be useful for us to borrow from. If I understand correctly, they did it by subpackage over a transition period (see step 4 in their [Formatting Code with Black](https://github.com/astropy/astropy-APEs/blob/main/APE20.rst#implementation) proposal). If we could do this module by module perhaps starting with a small one that might not have any PRs or a few PRs opened for it then we can avoid the issue of this causing huge diffs all at once. I am assuming also that we can configure `pre-commit` to run on only on certain subdirectories.
Thanks for the awesome research and ideas and not chewing me out for not doing this in all this time!
Wrote a small script to identify files that are currently touched by PRs.

This could help identify which part of the codebase are safe to start reformating (in terms of merge conflict).

https://gist.github.com/Remi-Gau/6194563cabcfb1b89ce885a4f762ecbc

<details>
    <summary>files touched by PRs (2023/02/20)</summary>
<pre>
.github/workflows/auto-comment.yml
.github/workflows/update_precommit_hooks.yml
doc/Makefile
doc/changes/0.9.0.rst
doc/changes/0.9.1.rst
doc/changes/latest.rst
doc/changes/names.rst
doc/glm/first_level_model.rst
doc/introduction.rst
doc/manipulating_images/input_output.rst
doc/modules/datasets.rst
doc/modules/reference.rst
doc/references.bib
doc/themes/nilearn/static/nature.css_t
doc/whats_new.rst
examples/01_plotting/plot_3d_map_to_surface_projection.py
examples/01_plotting/plot_demo_glass_brain.py
examples/01_plotting/plot_demo_glass_brain_extensive.py
examples/01_plotting/plot_demo_more_plotting.py
examples/01_plotting/plot_demo_plotting.py
examples/01_plotting/plot_surf_atlas.py
examples/01_plotting/plot_surf_stat_map.py
examples/01_plotting/plot_surface_projection_strategies.py
examples/02_decoding/plot_haxby_frem.py
examples/02_decoding/plot_haxby_searchlight_surface.py
examples/03_connectivity/plot_compare_shrunk_connectome.py
examples/03_connectivity/plot_group_level_connectivity.py
examples/04_glm_first_level/plot_fiac_analysis.py
examples/04_glm_first_level/plot_fixed_effects.py
examples/04_glm_first_level/plot_localizer_surface_analysis.py
examples/04_glm_first_level/plot_two_session_model.py
examples/07_advanced/plot_surface_bids_analysis.py
nilearn/_utils/__init__.py
nilearn/_utils/docs.py
nilearn/_utils/helpers.py
nilearn/connectome/__init__.py
nilearn/connectome/population_shrunk_covariance.py
nilearn/coordimage/caretspec.py
nilearn/coordimage/coordimage.py
nilearn/coordimage/pointset.py
nilearn/coordimage/tests/__init__.py
nilearn/coordimage/tests/data/fsLR.wb.spec
nilearn/coordimage/tests/test_coordimage.py
nilearn/coordimage/tests/test_pointset.py
nilearn/datasets/__init__.py
nilearn/datasets/atlas.py
nilearn/datasets/data/image_10426.nii.gz
nilearn/datasets/description/fsaverage.rst
nilearn/datasets/description/fsaverage3.rst
nilearn/datasets/description/fsaverage4.rst
nilearn/datasets/description/fsaverage5.rst
nilearn/datasets/description/fsaverage6.rst
nilearn/datasets/struct.py
nilearn/datasets/tests/test_atlas.py
nilearn/datasets/tests/test_utils.py
nilearn/datasets/utils.py
nilearn/glm/contrasts.py
nilearn/glm/first_level/first_level.py
nilearn/glm/first_level/hemodynamic_models.py
nilearn/glm/tests/test_contrasts.py
nilearn/glm/tests/test_first_level.py
nilearn/glm/tests/test_second_level.py
nilearn/input_data/base_surf_masker.py
nilearn/input_data/nifti_labels_masker.py
nilearn/input_data/nifti_spheres_masker.py
nilearn/input_data/surf_labels_masker.py
nilearn/input_data/tests/test_surf_labels_masker.py
nilearn/interfaces/fmriprep/data/test-v21_desc-confounds_timeseries.json
nilearn/interfaces/fmriprep/data/test-v21_desc-confounds_timeseries.tsv
nilearn/interfaces/fmriprep/load_confounds_compcor.py
nilearn/interfaces/fmriprep/tests/test_load_confounds.py
nilearn/interfaces/fmriprep/tests/test_load_confounds_strategy.py
nilearn/interfaces/fmriprep/tests/test_load_confounds_utils.py
nilearn/interfaces/fmriprep/tests/utils.py
nilearn/maskers/multi_nifti_labels_masker.py
nilearn/maskers/multi_nifti_maps_masker.py
nilearn/maskers/nifti_spheres_masker.py
nilearn/plotting/data/html/surface_plot_template.html
nilearn/plotting/data/js/surface-plot-utils.js
nilearn/plotting/displays.py
nilearn/plotting/displays/_axes.py
nilearn/plotting/glass_brain.py
nilearn/plotting/html_surface.py
nilearn/plotting/img_plotting.py
nilearn/plotting/surf_plotting.py
nilearn/plotting/tests/test_html_surface.py
nilearn/plotting/tests/test_img_plotting.py
nilearn/plotting/tests/test_surf_plotting.py
nilearn/regions/signal_extraction.py
nilearn/regions/tests/test_signal_extraction.py
nilearn/reporting/data/html/report_body_template_niftispheresmasker.html
nilearn/reporting/html_report.py
nilearn/reporting/tests/test_html_report.py
nilearn/signal.py
nilearn/surface/__init__.py
nilearn/surface/nisurf.py
nilearn/surface/surface.py
nilearn/surface/tests/test_nisurf.py
nilearn/surface/tests/test_surface.py
nilearn/tests/test_common.py
nilearn/tests/test_signal.py
</pre>
</details>
@Remi-Gau thanks this is great! From looking at the list I can see that potential modules to start with would be either `nilearn/decoding` or `nilearn/decomposition`. The latter has less lines of code so let's take this first. So the steps would be:

1. Black all python files in `nilearn/decomposition` and open a PR
2. Set up pre-commit to work only on this module
3. Set up a flake8 CI workflow to run on entire python files, again only for this module
4. Once PR is reviewed and merged we can move on to the next module and repeat the steps

@kchawla-pi and others WDYT?
Sounds like a sane divide and conquer approach
@kchawla-pi are you ok if I give this a go or would I be reinventing your wheel?
> @kchawla-pi are you ok if I give this a go or would I be reinventing your wheel?

By all means, go ahead.
@Remi-Gau and thanks for that awesome gir script!
As I am testing things on my fork for #3484, [sourcery](https://sourcery.ai/) is suggesting to me some refactoring.

https://github.com/Remi-Gau/nilearn/pull/3/files

As the main target of those refactoring I see are f-strings, it made me think that as the whole code base will have to eventually be touched by this reformatting, it could be a good thing to add a hook in the pre-commit config to also update to more 'recent' python features by using pyugrade. 

https://github.com/asottile/pyupgrade


Will open a PR for the `nilearn/decoding` module
I updated list of files involved in PRs: https://github.com/nilearn/nilearn/issues/2528#issuecomment-1414489207

A) Nilearn modules that that are NOT currently in a PR
- nilearn/externals
- nilearn/image

B) What could also be done is to apply to start applying the pre-commit hooks to files in:
- the root of the repo
- build_tools
- maint_tools
- binder
- .cirlceci
- .github
- examples/05_...
- examples/06_...

For B it would start applying those hooks to non-python files:
  - id: check-json
  - id: check-merge-conflict
  - id: check-yaml
  - id: end-of-file-fixer
  - id: trailing-whitespace
  - id: codespell

Codespell is checked in CI and is therefore "enforced", so running early check with pre-commit makes sense.

`check-json`, `check-yaml` and `check-merge-conflict` ensure that we only commit parseable files.

`end-of-file-fixer` and `trailing-whitespace` tends to enforce some good practices on all text files.

Let me know your thoughts and what should be the next "target".


Another potential thing that can be automated in terms of formatting is sorting the imports.

If there is some interest in this, I could add an isort or reorder_python_imports pre-commit hook.

Tried playing around with isort but was running into some issues to make it play nice with black.

So an alternative is reorder_python_imports that enforces a single import per line (which also reduces the chances of merge conflicts)
https://github.com/asottile/reorder_python_imports#why-this-style
What's the problem with Black and isort interoperability?

On Fri, Feb 24, 2023, 03:54 Remi Gau ***@***.***> wrote:

> Another potential thing that can be automated in terms of formatting is
> sorting the imports.
>
> If there is some interest in this, I could add an isort or
> reorder_python_imports pre-commit hook.
>
> Tried playing around with isort but was running into some issues to make
> it play nice with black.
>
> So an alternative is reorder_python_imports that enforces a single import
> per line (which also reduces the chances of merge conflicts)
> https://github.com/asottile/reorder_python_imports#why-this-style
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/nilearn/nilearn/issues/2528#issuecomment-1443202554>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB6SXRAXISPKHJSFQU42PJLWZBZM5ANCNFSM4OHMSDXA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

> What's the problem with Black and isort interoperability?

Of course now that I want to demonstrate the issue, it works just fine. Yet another shy bug that won't show itself if there is more than one person looking at it.

Will a open a WIP PR to start implementing isort along with black.
update list of files in PRs (excluding the formatting PRs and those that only touch files already reformatted):

#3586, #3532, #3589, #3558, #3590, #3584, #3572, #3569

| file | nb of PR found in | new file |
| ---- | ----------------- | -------- |
| doc/changes/latest.rst | 6 | False |
| doc/changes/names.rst | 2 | False |
| doc/glm/first_level_model.rst | 1 | False |
| doc/introduction.rst | 1 | False |
| doc/maintenance.rst | 1 | False |
| doc/manipulating_images/input_output.rst | 1 | False |
| doc/modules/reference.rst | 2 | True |
| doc/references.bib | 1 | False |
| doc/whats_new.rst | 4 | True |
| examples/00_tutorials/plot_3d_and_4d_niimg.py | 1 | False |
| examples/01_plotting/plot_3d_map_to_surface_projection.py | 2 | False |
| examples/01_plotting/plot_demo_plotting.py | 1 | False |
| examples/01_plotting/plot_multiscale_parcellations.py | 1 | False |
| examples/01_plotting/plot_prob_atlas.py | 1 | False |
| examples/01_plotting/plot_surf_atlas.py | 2 | False |
| examples/01_plotting/plot_surf_stat_map.py | 1 | False |
| examples/01_plotting/plot_surface_projection_strategies.py | 1 | False |
| examples/02_decoding/plot_haxby_searchlight_surface.py | 1 | False |
| examples/03_connectivity/plot_compare_shrunk_connectome.py | 1 | True |
| examples/03_connectivity/plot_group_level_connectivity.py | 1 | False |
| examples/04_glm_first_level/plot_fiac_analysis.py | 1 | False |
| examples/04_glm_first_level/plot_fixed_effects.py | 1 | False |
| examples/04_glm_first_level/plot_localizer_surface_analysis.py | 1 | False |
| examples/04_glm_first_level/plot_two_session_model.py | 1 | True |
| examples/07_advanced/plot_surface_bids_analysis.py | 1 | False |
| nilearn/_utils/__init__.py | 2 | False |
| nilearn/_utils/bids.py | 1 | True |
| nilearn/_utils/data_gen.py | 1 | False |
| nilearn/_utils/docs.py | 3 | False |
| nilearn/_utils/helpers.py | 1 | False |
| nilearn/connectome/__init__.py | 1 | False |
| nilearn/connectome/population_shrunk_covariance.py | 1 | True |
| nilearn/coordimage/caretspec.py | 1 | True |
| nilearn/coordimage/coordimage.py | 1 | True |
| nilearn/coordimage/pointset.py | 1 | True |
| nilearn/coordimage/tests/__init__.py | 1 | True |
| nilearn/coordimage/tests/data/fsLR.wb.spec | 1 | True |
| nilearn/coordimage/tests/test_coordimage.py | 1 | True |
| nilearn/coordimage/tests/test_pointset.py | 1 | True |
| nilearn/datasets/description/fsaverage.rst | 1 | False |
| nilearn/datasets/description/fsaverage3.rst | 1 | False |
| nilearn/datasets/description/fsaverage4.rst | 1 | False |
| nilearn/datasets/description/fsaverage5.rst | 1 | False |
| nilearn/datasets/description/fsaverage6.rst | 1 | False |
| nilearn/datasets/struct.py | 1 | False |
| nilearn/datasets/tests/test_func.py | 1 | False |
| nilearn/decomposition/dict_learning.py | 1 | False |
| nilearn/glm/contrasts.py | 1 | False |
| nilearn/glm/first_level/first_level.py | 1 | False |
| nilearn/glm/first_level/hemodynamic_models.py | 1 | False |
| nilearn/glm/tests/test_contrasts.py | 1 | False |
| nilearn/glm/tests/test_first_level.py | 2 | False |
| nilearn/glm/tests/test_hemodynamic_models.py | 1 | False |
| nilearn/glm/tests/test_second_level.py | 2 | False |
| nilearn/input_data/base_surf_masker.py | 1 | True |
| nilearn/input_data/nifti_labels_masker.py | 1 | False |
| nilearn/input_data/nifti_spheres_masker.py | 1 | False |
| nilearn/input_data/surf_labels_masker.py | 1 | True |
| nilearn/input_data/tests/test_surf_labels_masker.py | 1 | True |
| nilearn/interfaces/fmriprep/data/test-v21_desc-confounds_timeseries.json | 1 | True |
| nilearn/interfaces/fmriprep/data/test-v21_desc-confounds_timeseries.tsv | 1 | True |
| nilearn/interfaces/fmriprep/load_confounds_compcor.py | 1 | False |
| nilearn/interfaces/fmriprep/tests/test_load_confounds.py | 1 | False |
| nilearn/interfaces/fmriprep/tests/test_load_confounds_strategy.py | 1 | False |
| nilearn/interfaces/fmriprep/tests/test_load_confounds_utils.py | 1 | False |
| nilearn/interfaces/fmriprep/tests/utils.py | 1 | False |
| nilearn/maskers/multi_nifti_labels_masker.py | 1 | False |
| nilearn/maskers/multi_nifti_maps_masker.py | 1 | False |
| nilearn/maskers/nifti_spheres_masker.py | 1 | False |
| nilearn/plotting/data/html/surface_plot_template.html | 1 | False |
| nilearn/plotting/data/js/surface-plot-utils.js | 1 | False |
| nilearn/plotting/displays.py | 2 | True |
| nilearn/plotting/displays/_axes.py | 1 | False |
| nilearn/plotting/glass_brain.py | 1 | False |
| nilearn/plotting/html_surface.py | 4 | False |
| nilearn/plotting/img_plotting.py | 2 | False |
| nilearn/plotting/surf_plotting.py | 5 | False |
| nilearn/plotting/tests/test_html_document.py | 1 | False |
| nilearn/plotting/tests/test_html_surface.py | 2 | False |
| nilearn/plotting/tests/test_img_plotting.py | 1 | True |
| nilearn/plotting/tests/test_matplotlib_backend.py | 1 | False |
| nilearn/plotting/tests/test_surf_plotting.py | 2 | False |
| nilearn/regions/tests/test_parcellations.py | 1 | False |
| nilearn/regions/tests/test_signal_extraction.py | 1 | False |
| nilearn/reporting/data/html/report_body_template_niftispheresmasker.html | 1 | True |
| nilearn/reporting/html_report.py | 1 | False |
| nilearn/reporting/tests/test_html_report.py | 1 | False |
| nilearn/signal.py | 1 | False |
| nilearn/surface/__init__.py | 1 | False |
| nilearn/surface/nisurf.py | 1 | True |
| nilearn/surface/surface.py | 2 | False |
| nilearn/surface/tests/test_nisurf.py | 1 | True |
| nilearn/surface/tests/test_surface.py | 1 | False |
| nilearn/tests/test_common.py | 1 | True |
| nilearn/tests/test_signal.py | 2 | False |

You can do this in the pre-commit configuration so you don't have to manually address potential conflicts:

```yaml
  - repo: https://github.com/PyCQA/isort
    rev: 5.12.0
    hooks:
      - id: isort
        args: ["--profile", "black"]
```
With a bunch of PRs merged today, what's the outlook above for the number of affected files?
> You can do this in the pre-commit configuration so you don't have to manually address potential conflicts:
> 
> ```yaml
>   - repo: https://github.com/PyCQA/isort
>     rev: 5.12.0
>     hooks:
>       - id: isort
>         args: ["--profile", "black"]
> ```

isn't that already the case?
> With a bunch of PRs merged today, what's the outlook above for the number of affected files?

need to update the table
file list updated: https://github.com/nilearn/nilearn/issues/2528#issuecomment-1451860246
Will these pre-commit hooks affect non *.py files? If not, they can be eliminated from this list.
Pre commit will for example remove trailing spaces from ANY file. So some of them would be affected.

But you are right that the files that are already under pre commit purview should be omitted from the list
The file listing should omit those that pre-commit takes of already
Having a look at what module to tackle next, I come across #1994 that seems stalled and where a lot of the changes are actually in new files so the risk of merge conflict if this module was reformatted would be lower. 
I agree you can probably take on the connectome module now. That PR most likely has to be superseded by one of the maintainers
Trying to see what thing to tackle next.
Maybe working on the tests of some modules would be an option.
Would also make for some smaller prs so less annoying to review. 

Resolving patch:
diff --git a/examples/01_plotting/plot_surf_atlas.py b/examples/01_plotting/plot_surf_atlas.py
--- a/examples/01_plotting/plot_surf_atlas.py
+++ b/examples/01_plotting/plot_surf_atlas.py
@@ -35,12 +35,12 @@
 fsaverage = datasets.fetch_surf_fsaverage()
 
 # The fsaverage dataset contains file names pointing to the file locations
-print('Fsaverage5 pial surface of left hemisphere is at: %s' %
-      fsaverage['pial_left'])
-print('Fsaverage5 inflated surface of left hemisphere is at: %s' %
-      fsaverage['infl_left'])
-print('Fsaverage5 sulcal depth map of left hemisphere is at: %s' %
-      fsaverage['sulc_left'])
+print("Fsaverage5 pial surface of left hemisphere is at: "
+      f"{fsaverage['pial_left']}")
+print("Fsaverage5 inflated surface of left hemisphere is at: "
+      f"{fsaverage['infl_left']}")
+print("Fsaverage5 sulcal depth map of left hemisphere is at: "
+      f"{fsaverage['sulc_left']}")
 
 ###############################################################################
 # Visualization
diff --git a/nilearn/_utils/data_gen.py b/nilearn/_utils/data_gen.py
--- a/nilearn/_utils/data_gen.py
+++ b/nilearn/_utils/data_gen.py
@@ -677,7 +677,7 @@ def generate_group_sparse_gaussian_graphs(n_subjects=5,
     topology = topology > 0
     assert (np.all(topology == topology.T))
     logger.log(
-        "Sparsity: {0:f}".format(1. * topology.sum() / (topology.shape[0]**2)),
+        f"Sparsity: {1.0 * topology.sum() / topology.shape[0] ** 2 :f}",
         verbose=verbose)
 
     # Generate temporal signals
diff --git a/nilearn/_utils/docs.py b/nilearn/_utils/docs.py
--- a/nilearn/_utils/docs.py
+++ b/nilearn/_utils/docs.py
@@ -902,6 +902,5 @@ def fill_doc(f):
     except (TypeError, ValueError, KeyError) as exp:
         funcname = f.__name__
         funcname = docstring.split('\n')[0] if funcname is None else funcname
-        raise RuntimeError('Error documenting %s:\n%s'
-                           % (funcname, str(exp)))
+        raise RuntimeError(f'Error documenting {funcname}:\n{str(exp)}')
     return f


Test patch: