Repository: pytest-dev/pytest
Pull request: Introduce guidelines for closing stale issues/PRs
Close #7282


(cc @gnikonorov which also participated in the discussion)

Problem statement:
Proposal on how to deal with stale PRs
Hi everyone,

Currently we have a bunch of PRs open (50+), and I notice many of them are "stale", meaning that we have asked for changes and the authors didn't got around to implement them yet after a somewhat long time.

There are many reasons why people don't implement changes: they might get busy, lost interest in a particular issue, or plainly just forgot about it.

In the past we used to send "gentle pings" to authors, but this does not scale very well: we still need to go back to the large list of open PRs periodically to identify stale ones where we have already sent a ping to users, before deciding to close it because of lack of response. And this is a bit tiring, because often you need to re-read a whole thread to identify if a PR is stale or waiting for an action on our part.

So I would like to reduce the large number of open PRs, because having this large a list makes it easy to lose some notifications about changes.

So in order to do that, I propose: from now on we close PRs which we have identified as stale right away, leaving a nice message to the user explaining that we really appreciate their efforts, but we want to close the PRs to clean off our queue, and encouraging them to re-open the PR once they get back to it. This is just to be meant as a gentle reminder, because users can easily re-open an existing PR at the click of a button. Of course we need to write a nicely worded message so users, which took time and effort to contribute, don't feel offended or brushed off.

Just to make sure I get my point across, here's an example of a message I think would be appropriate while closing a stale PR:

---

Hi <user>,

First of all we would like to thank you for your time and effort on working on this, the pytest team deeply appreciates it.

We noticed it has been awhile since you have updated this PR, however. pytest is a high activity project, with may issues/PRs being opened daily, so it is hard for us maintainers to track which PRs are ready for merging, for review, or need more attention. 

So for those reasons we think it is best to close the PR for now, but with the only intention to cleanup our queue, it is by no means a rejection of your changes. We still encourage you to re-open this PR (it is just a click of a button away) when you are ready to get back to it.

Again we appreciate your time for working on this, and hope you might get back to this at a later time!

Thanks, the pytest team.

--- 

Thoughts?

Hints from issue discussion:
I like the idea, it would be nice to have a bot that manages staleness, pings and closes 
There's https://github.com/probot/stale, which has all configurations we need: customize number of days for ping, closing, and limiting only to PRs.
Any chance this can be added to issues too? I see that it has support for issues, and speaking as someone that is brand new to trying to contributing to pytest, the large issue list is very confusing as it would appear that many may be abandoned/out of date.
Hi @gnikonorov,

> Any chance this can be added to issues too?

I'm not sure, there's a lot of issues in the tracker that are still relevant and should be worked on eventually. But I agree it is hard to navigate the tracker and identify what's relevant still or not.
@nicoddemus I see prbot has the config option `exemptLabels` which makes it ignore anything with that label. Would it be too much to ask issue responders/issue submitters to mark the issue with a label in `exemptLabels` if it is a genuine bug? We can also configure prbot to ignore issues with an assignee.

I know this would be a headache for existing issues, but on the plus side it would help clean up the issue list.

Just a thought, I know this is a big ask time commitment wise from the core development team
I like the idea. However, there is another point one should talk about: 
There are PRs, which are nearly finished, so only a few changes need to be made, but the author does not get to it/respond or whatever. Would it be an option to finish these PRs and merge them if the effort is not that huge? 

The benefit would be that the work is not lost and the original author is credited, too.
Good idea +1. Would also like to see more marking of ‘easy’ issues as that’s where people tend to start -maybe there haven’t been that many recently but the marker seems somewhat neglected 
> There are PRs, which are nearly finished, so only a few changes need to be made, but the author does not get to it/respond or whatever. Would it be an option to finish these PRs and merge them if the effort is not that huge?

Oh definitely, thanks for bringing that up.

> Would also like to see more marking of ‘easy’ issues as that’s where people tend to start -maybe there haven’t been that many recently but the marker seems somewhat neglected

You are right, sometimes it seems we miss applying that label. 👍 

I do agree that we should close stale things, but I'd rather see us do it as humans and not use stalebot.  personally I find stalebot to be kind of a rude way to handle issues (tedious for users to ping-open their tickets, really annoying if you turn on locking as well).
> but I'd rather see us do it as humans and not use stalebot.

Are you talking about issues, PRs or both?

For PRs, I'm OK with that too, because it is easy to filter by "update date" (say last updated 30 days ago) and close them with a template message.

But for issues that does not scale unfortunately.

Commit message: Introduce guidelines for closing stale issues/PRs

Close #7282

Patch to review:
diff --git a/.coveragerc b/.coveragerc
deleted file mode 100644
index b06629a8a8f..00000000000
--- a/.coveragerc
+++ /dev/null
@@ -1,29 +0,0 @@
-[run]
-include =
-  src/*
-  testing/*
-  */lib/python*/site-packages/_pytest/*
-  */lib/python*/site-packages/pytest.py
-  */pypy*/site-packages/_pytest/*
-  */pypy*/site-packages/pytest.py
-  *\Lib\site-packages\_pytest\*
-  *\Lib\site-packages\pytest.py
-parallel = 1
-branch = 1
-
-[paths]
-source = src/
-  */lib/python*/site-packages/
-  */pypy*/site-packages/
-  *\Lib\site-packages\
-
-[report]
-skip_covered = True
-show_missing = True
-exclude_lines =
-    \#\s*pragma: no cover
-    ^\s*raise NotImplementedError\b
-    ^\s*return NotImplemented\b
-    ^\s*assert False(,|$)
-
-    ^\s*if TYPE_CHECKING:
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 3d07db63804..836b4b29552 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -289,7 +289,7 @@ Here is a simple overview, with pytest-specific bits:
 
 
 Writing Tests
-----------------------------
+~~~~~~~~~~~~~
 
 Writing tests for plugins or for pytest itself is often done using the `testdir fixture <https://docs.pytest.org/en/latest/reference.html#testdir>`_, as a "black-box" test.
 
@@ -328,6 +328,19 @@ one file which looks like a good fit. For example, a regression test about a bug
 should go into ``test_cacheprovider.py``, given that this option is implemented in ``cacheprovider.py``.
 If in doubt, go ahead and open a PR with your best guess and we can discuss this over the code.
 
+Joining the Development Team
+----------------------------
+
+Anyone who has successfully seen through a pull request which did not
+require any extra work from the development team to merge will
+themselves gain commit access if they so wish (if we forget to ask please send a friendly
+reminder).  This does not mean there is any change in your contribution workflow:
+everyone goes through the same pull-request-and-review process and
+no-one merges their own pull requests unless already approved.  It does however mean you can
+participate in the development process more fully since you can merge
+pull requests from other contributors yourself after having reviewed
+them.
+
 
 Backporting bug fixes for the next patch release
 ------------------------------------------------
@@ -359,15 +372,49 @@ actual latest release). The procedure for this is:
    * Delete the PR body, it usually contains a duplicate commit message.
 
 
-Joining the Development Team
-----------------------------
+Handling stale issues/PRs
+-------------------------
 
-Anyone who has successfully seen through a pull request which did not
-require any extra work from the development team to merge will
-themselves gain commit access if they so wish (if we forget to ask please send a friendly
-reminder).  This does not mean there is any change in your contribution workflow:
-everyone goes through the same pull-request-and-review process and
-no-one merges their own pull requests unless already approved.  It does however mean you can
-participate in the development process more fully since you can merge
-pull requests from other contributors yourself after having reviewed
-them.
+Stale issues/PRs are those where pytest contributors have asked for questions/changes
+and the authors didn't got around to answer/implement them yet after a somewhat long time, or
+the discussion simply died because people seemed to lost interest.
+
+There are many reasons why people don't answer questions or implement requested changes:
+they might get busy, lost interest, or just forgot about it,
+but the fact is that this is very common in open source software.
+
+The pytest team really appreciates every issue and pull request, but being a high-volume project
+with many issues and pull requests being submitted daily, we try to reduce the number of stale
+issues and PRs by regularly closing them. When an issue/pull request is closed in this manner,
+it is by no means a dismissal of the topic being tackled by the issue/pull request, but it
+is just a way for us to clear up the queue and make the maintainers work more manageable. Submitters
+can always reopen the issue/pull request in their own time later if it makes sense.
+
+When to close
+~~~~~~~~~~~~~
+
+Here are a few general rules the maintainers use to decide when to close issues/PRs because
+of lack of inactivity:
+
+* Issues labeled ``question``: closed after 14 days inactive.
+* Issues labeled ``proposal``: closed after six months inactive.
+* Pull requests: after one month, consider pinging the author, update linked issue, or consider closing. For pull requests which are nearly finished, the team should consider finishing it up and merging it.
+
+The above are **not hard rules**, but merely **guidelines**, and can be (and often are!) reviewed on a case-by-case basis.
+
+Closing pull requests
+~~~~~~~~~~~~~~~~~~~~~
+
+When closing a Pull Request, it needs to be acknowledge the time, effort, and interest demonstrated by the person which submitted it. As mentioned previously, it is not the intent of the team to dismiss stalled pull request entirely but to merely to clear up our queue, so a message like the one below is warranted when closing a pull request that went stale:
+
+    Hi <contributor>,
+
+    First of all we would like to thank you for your time and effort on working on this, the pytest team deeply appreciates it.
+
+    We noticed it has been awhile since you have updated this PR, however. pytest is a high activity project, with may issues/PRs being opened daily, so it is hard for us maintainers to track which PRs are ready for merging, for review, or need more attention.
+
+    So for those reasons we think it is best to close the PR for now, but with the only intention to cleanup our queue, it is by no means a rejection of your changes. We still encourage you to re-open this PR (it is just a click of a button away) when you are ready to get back to it.
+
+    Again we appreciate your time for working on this, and hope you might get back to this at a later time!
+
+    <bye>


Code review comments:
diff_hunk: @@ -359,15 +372,49 @@ actual latest release). The procedure for this is:
    * Delete the PR body, it usually contains a duplicate commit message.
 
 
-Joining the Development Team
-----------------------------
+Handling stale issues/PRs
+-------------------------
 
-Anyone who has successfully seen through a pull request which did not
-require any extra work from the development team to merge will
-themselves gain commit access if they so wish (if we forget to ask please send a friendly
-reminder).  This does not mean there is any change in your contribution workflow:
-everyone goes through the same pull-request-and-review process and
-no-one merges their own pull requests unless already approved.  It does however mean you can
-participate in the development process more fully since you can merge
-pull requests from other contributors yourself after having reviewed
-them.
+Stale issues/PRs are those where pytest contributors have asked for questions/changes
+and the authors didn't got around to answer/implement them yet after a somewhat long time, or
+the discussion simply died because people seemed to lost interest.
path: CONTRIBUTING.rst
text: @user1:
```suggestion
the discussion simply died because people seemed to lose interest.
```

diff_hunk: @@ -359,15 +372,49 @@ actual latest release). The procedure for this is:
    * Delete the PR body, it usually contains a duplicate commit message.
 
 
-Joining the Development Team
-----------------------------
+Handling stale issues/PRs
+-------------------------
 
-Anyone who has successfully seen through a pull request which did not
-require any extra work from the development team to merge will
-themselves gain commit access if they so wish (if we forget to ask please send a friendly
-reminder).  This does not mean there is any change in your contribution workflow:
-everyone goes through the same pull-request-and-review process and
-no-one merges their own pull requests unless already approved.  It does however mean you can
-participate in the development process more fully since you can merge
-pull requests from other contributors yourself after having reviewed
-them.
+Stale issues/PRs are those where pytest contributors have asked for questions/changes
+and the authors didn't got around to answer/implement them yet after a somewhat long time, or
+the discussion simply died because people seemed to lost interest.
+
+There are many reasons why people don't answer questions or implement requested changes:
+they might get busy, lost interest, or just forgot about it,
path: CONTRIBUTING.rst
text: @user1:
```suggestion
they might get busy, lose interest, or just forget about it,
```

diff_hunk: @@ -359,15 +372,49 @@ actual latest release). The procedure for this is:
    * Delete the PR body, it usually contains a duplicate commit message.
 
 
-Joining the Development Team
-----------------------------
+Handling stale issues/PRs
+-------------------------
 
-Anyone who has successfully seen through a pull request which did not
-require any extra work from the development team to merge will
-themselves gain commit access if they so wish (if we forget to ask please send a friendly
-reminder).  This does not mean there is any change in your contribution workflow:
-everyone goes through the same pull-request-and-review process and
-no-one merges their own pull requests unless already approved.  It does however mean you can
-participate in the development process more fully since you can merge
-pull requests from other contributors yourself after having reviewed
-them.
+Stale issues/PRs are those where pytest contributors have asked for questions/changes
+and the authors didn't got around to answer/implement them yet after a somewhat long time, or
path: CONTRIBUTING.rst
text: @user1:
```suggestion
and the authors didn't get around to answer/implement them yet after a somewhat long time, or
```