PDEP-1 Revision (Decision Making)
State: closed

The governance working group has been discussing the PDEP workflow and decision-making for some time now. **Feedback is welcome.**

This PR proposes some formulation of the existing workflow to clarify and facilitate it for current and future authors, and voting members.
In our discussions, we assessed that a more structured process would help folks who submit the PDEP know how and when it can progress, and how to better engage with others.

The timelines were considered to facilitate an engaging discussion, and automation. We aim to make the lives of the PDEP authors easier by taking away the need to decide "has it been long enough", or "cat herding" folks to engage and clarify their comments so that PDEPs don't get large change requests that could have been clarified early on - late in the process.

The quorum was discussed a lot in our working group, and we're unsure about the best strategy here. We were looking for a sufficient number so if a decision is passed one feels it's valid (e.g. maybe 2 people is too low), but not too high so that we feel like ongoing discussions are blocked because it's too hard to get that many voting members involved. Further proposal on this point are welcome.

We tried to make things as flexible and clear as possible. Being mindful of the needs of the authors, as well as those engaging in the discussion, be them voting members or not.

At this point, we would like to invite feedback. It might be that you need more clarity on the suggestions, have ideas for improvement, or aren't convinced that this will affect the current process positively.

Co-authored-by: @MarcoGorelli, @rhshadrach, @Dr-Irv, @jorisvandenbossche

Labels:
name: Stale
name: PDEP
description: pandas enhancement proposal

Comments:
+1 from me.
@pandas-dev/pandas-core are there more concerns or comments on the proposed voting procedure? 
Curveball: I have been thinking about the voting majority a lot, and I am leaning toward making it a simple majority or 60% at most (e.g. 7/11 approving votes). 
My thinking is that I would like to see interesting and important discussions that help shape the outcome, but I don't want decisions to be hard, actually. 
My caution is that if many decisions pass this way, we might need to re-evaluate because it can alienate anyone who is consistently feeling like they have no say. But, I hope that the discussion period will, in most cases, gives time for authors to reconsider their proposals based on discussion - even if they "have enough votes". 
Do folks want to discuss this further in the next governance meeting?
I agree with @noatamir, 75% is too high for me to feel comfortable
I disagree. I recall the arguments made in the original governance meeting. With 75%, a 3:1 vote passes and a PDEP is accepted. With 60% 2 in 5 people can oppose and a PDEP still be accepted. For a project which is expected to have a very clear and defined development structure a 3:2 vote is too divided and should not be accepted, IMO.
> My caution is that if many decisions pass this way, we might need to re-evaluate because it can alienate anyone who is consistently feeling like they have no say.

I think the opposite could be true.

From a recent personal perspective, I felt that the support for the pyarrow dependency was likely to be so strong that I was happy not to challenge the proposal and only if it failed to get through would get involved to help shape the proposal to something acceptable to all.

In order to downvote I was 'required' to participate. And to be fair, the proposal was amended following my comments and additional discussion and I am now more comfortable with it.

But OTH, I was also happy to proceed with a vote as there was maybe a chance that even with 75% there was a chance it would have been accepted and therefore these amendments and further discussion has potentially slowed the process.

So I would argue the opposite. We need a higher threshold so that people feel they have a say.

But I appreciate that if we go much higher, we could effectively return to the veto system. 
> With 75%, a 3:1 vote passes and a PDEP is accepted. With 60% 2 in 5 people can oppose and a PDEP still be accepted. For a project which is expected to have a very clear and defined development structure a 3:2 vote is too divided and should not be accepted, IMO.

4:1 is 80%, 5:1 is 83.33% and 6:1 is 85.7%

5:1 means that if 2 people oppose a proposal, we need at least 12 members to vote or change the proposal in order to be accepted.

So 5:1 could lead to difficult discussions/slow progress, but increasing to 80% (4:1) does not seem unreasonable to me.




While I agree that 3-2 is not a great result, I don't think that this is a realistic outcome. We require at least 11 votes for a PDEP to possibly get accepted. I am not comfortable with a 8-3 result not being sufficient to pass a PDEP, same for 11-4. Personally, I feel that both of these results should be strong enough majorities to accept a PDEP
> While I agree that 3-2 is not a great result, I don't think that this is a realistic outcome. We require at least 11 votes for a PDEP to possibly get accepted. I am not comfortable with a 8-3 result not being sufficient to pass a PDEP, same for 11-4. Personally, I feel that both of these results should be strong enough majorities to accept a PDEP

I agree.  That's why I felt good about the 75% rule.  Going above that seemed to high, below seemed to low.  Our current policy of 100% isn't sustainable.
> > While I agree that 3-2 is not a great result, I don't think that this is a realistic outcome. We require at least 11 votes for a PDEP to possibly get accepted. I am not comfortable with a 8-3 result not being sufficient to pass a PDEP, same for 11-4. Personally, I feel that both of these results should be strong enough majorities to accept a PDEP
> 
> I agree. That's why I felt good about the 75% rule. Going above that seemed to high, below seemed to low. Our current policy of 100% isn't sustainable.

But 75% would not be sufficient to pass a 8-3 or 11-4, which feels off to me
> But 75% would not be sufficient to pass a 8-3 or 11-4, which feels off to me

Maybe we lower it to 70% ?


TBH I'd be OK with 11-4 not passing. Perhaps we can go ahead with what's been agreed upon, and then revise if there are many close calls like that?
Since there is already a higher barrier to disapproving(you have to engage in the PDEP discussion first), I think the current limit is fine.

We can always revisit this if necessary, in the future. I agree with Marco there.
Thanks for the responses, folks!
I was expecting a higher number of votes in most cases, considering the quorum requirement.  I don't anticipate that we will have many cases of 8 abstains and a 2-1 split.
Likewise, I agree that we can see if we encounter frustrations and make changes in either approach. 

One last question, is anyone apposed to me adding a requirement of minimum discussion time? I was thinking that 3–4 weeks would be the minimum before an early vote can be triggered. This was intended to support the lower majority threshold, but I think it fits in any case. 
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
@noatamir what do we need to close this out?
probably controversial, but closing as stale!
Yeah you are correct, I am -1 on closing
almost 6 months without a response to what is needed to close this out.

@phofl can you answer that?
As far as I remember, the main point of contention last summer was the exact threshold to require for acceptance. The text currently says "a majority of 75% of the non-abstaining votes", but there were some in favor of lowering this threshold. 

I think the main relevant discussion on this PR starts at https://github.com/pandas-dev/pandas/pull/53576#issuecomment-1631555955, continuing in the comments below.

I started a poll with the core team to get an idea of what threshold would be most acceptable.

Thanks @jorisvandenbossche 

Great to know the process hasn't been abandoned.

I think that the PDEP process (which is ironically this PR) should probably include processes for the scenarios where the PDEP process doesn't follow the suggested timescales and appears abandoned. This also applies to VOTE issues that is not closed after the voting period ends. (for instance do votes still count until the issue is closed?)

A PDEP or PDEP revision that is not explicitly rejected is also not accepted.

I think we need to be able to close stale PDEP PRs or voting issues rather than leaving open with zero communication so that it is clearer to the community what the status is.
@pandas-dev/pandas-core please take a final look / provide final comments, or otherwise (re-)approve the PR!

Main changes since last discussion round:
- Changed the threshold from 75% to 70%
- In the discussion timeline, tweaked the language around an early vote if there was no more discussion for 15 days (after a minimum of 30 days)

One more small change: I added an extra "Withdrawn" status in addition to "Rejected", for the case the author(s) themselves decide to not get to a vote if they are convinced from the discussion it's eventually not a good idea or when there is an alternative proposal that is decided to push forward, but we still want to merge the PDEP to document the discussion (this status is also used in PEPs, and also numpy and other projects).
A final reminder here, I plan to merge this by Friday, assuming we have consensus on those changes.
Thanks all. Seems like we have consensus here.
And thanks @noatamir and the initial write-up, and everyone for the feedback!