PDEP-1: Purpose and guidelines for pandas enhancement proposals
State: closed

Closes #28568

Initial PDEP to define purpose and guidelines for pandas enhancement proposals (equivalent to PEPs or NEPs). **Feedback very welcome**.

This PR also makes the PDEPs public in the [roadmap page](https://pandas.pydata.org/about/roadmap.html) of our website.


Labels:
name: Web
description: pandas website
name: PDEP
description: pandas enhancement proposal

Comments:
This would probably close https://github.com/pandas-dev/pandas/issues/28568
> I didn't immediately see this, but it would be good to explain how an accepted/rejected decision is made.

Very good point. I thought this was quite a big topic, and maybe more related to general governance than specific to PDEPs. That's why for now I decided to just link to the governance documentation. Based on the current governance, decisions are made by consensus, with the BDFL (Wes) having the final word when consensus is not possible.

I think the governance documentation is 7 years old. Many things changed since then, including the structure of the team (from like 3 active core devs to like 20, Python moved out of a BDFL model...). Also, besides the decision making, I think our governance seems outdated. I personally don't know most of the people in the pandas CoC committee, and I assume they are not performing any task, and probably don't even know they are part of it.

While I fully agree on what you say, I'd personally keep the discussions separate, as I think both are quiet big, and can be taken care of separately. What do you think?
I updated the PR to incorporate the feedback from comments, and I replied to the ones I didn't directly incorporate.

I also fixes couple of visualization issues, and improved a bit the styles, so the roadmap and also other pages are clearer.

Let me know of additional feedback, and feel free to approve when you're happy with getting this merged, or add a `Request changes` flagged if you see any blocker. Thanks!
@pandas-dev/pandas-core any other comment before merging this? If there are no objections I'll merge this in couple of days, and we can keep iterating as needed in separate PRs.
Related to my previous comment (https://github.com/pandas-dev/pandas/pull/47444#discussion_r905868612) about starting a new PDEP as draft / under discussion: this has been changed in the text, but at the same time we still say to create the new PDEP file in the ``web/pdeps/accepted/`` directory. 
Personally, I still find that a bit confusing and sending a potentially wrong message. If we have an `under_discussion` status, why not also reflect that in the directory / path? (although this probably also comes back to whether we have the intention to ever merge PRs in that status, because indeed if we never do that, we would actually never use that directory in practice)

Taking one step back: do we actually need those different directories? (I suppose they are currently used to simplify the script to automate the website content to list the PDEPs) 
For example, NumPy (and also Python PEPs) uses a flat directory for the documents: https://github.com/numpy/numpy/tree/main/doc/neps. And it then infers the status from the header. If we have a standard preamble that has the status in that list, that also sounds doable to automate this (instead of being based on directories)? It would make my above discussion point (about in which directory to start the proposal) moot, and can also simplify the overall process a little bit (eg you never need to move files for changing a status, but just change the Status line in the document (eg when a PDEP gets "implemented")).

Thanks all for the feedback, very interesting points.

For the voting, I'm working on the governance documents in parallel, and I'll propose changes there to have a way to decide when a PDEP is ready to merge.

For the name to use, there are several comments about names, and many valid points. It's probably not so important, but to find something that most people are happy with, I think we can run a quick poll. I'll write here different options including the ones already proposed. Feel free to suggest more, and when we're ready I'll send a link to the pandas-dev list with a poll.

- PDEP
- pdEP
- PEP
- EP
- PCP (pandas change proposal)
- DEP (dataframe enhancement proposal)

I'll be addressing the rest of the comments in this PR.
May I suggest adding a template for new proposals as part of this PR, or as a follow up if that's preferable? 😇 

I find templates useful to help facilitate structured discussions, and that important points relevant for consideration aren't missed by the proposal author(s) before submitting them, e.g. how would this affect backwards compatibility.  

For reference I include these two templates which could be relevant as a starting point:
- scikit-learn's [SLEP template](https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/slep_template.html)
- NumPy's [NEP template](https://numpy.org/neps/nep-template.html)
Thanks all for the feedback. Seems like the preferred name is PDEP, so keeping it. I addressed all the comments, there are two pending things, that I think make more sense to address in future PRs:

- How the exact decision on when a PDEP is approved (which is part of a wider discussion about governance)
- Adding a template for PDEPs. While this seems a great idea, I personally think it's premature to have it initially, and I'd wait until we've got few PDEPs, and then decide if we think it adds value, and what should be the format, with concrete examples

Let me know if any other comment. @jreback you reviewed with `Requested changes`, I rewrote that whole section that other people also found misleading, I think your comments should be addressed now.
Thanks all for the reviews and feedback!
An unexpected & pleasant surprise that this old issue of mine got fixed. Not sure I have the time/energy to pour those old PDEP-sized ideas into documents, but glad to know that there'd now be a way to do it.
Thanks @datapythonista for the PR.

> An unexpected & pleasant surprise that this old issue of mine got fixed. Not sure I have the time/energy to pour those old PDEP-sized ideas into documents, but glad to know that there'd now be a way to do it.

Thanks @h-vetinari. feel free to comment on those issues that a PDEP may be a good idea as others may think that they are worth pushing now.


and if you have already put a lot of time and effort in the discussion, you may have already made it alot easier to progress.