AI tools like Copilot Coding Agent don't know about / don't respect our Automated Contributions Policy

(I am creating an issue to a PR already opened (#31643), because there are many more ways to solve the problem.)

AI tools many people use to create PRs don't care about our [Automated Contributions Policy](https://scikit-learn.org/dev/developers/contributing.html#automated-contributions-policy). 

Since [GitHub Copilot Coding Agent Has Arrived!](https://github.com/orgs/community/discussions/159068) and people build [Github-MCP](https://github.com/dhyeyinf/Github-MCP)s that can be integrated with LLM clients, scikit-learn and other open source projects get an increasing amount of AI spam. Many people who care about open source are unhappy about it and request an option to block AI-generated PRs and issues on their projects ([Allow us to block Copilot-generated issues (and PRs) from our own repositories](https://github.com/orgs/community/discussions/159749)) - so far without success.

You can see that there is an increasing amount of partially or fully generated PRs and a decrease in overall quality for PRs on scikit-learn by looking at [the last closed PRs](https://github.com/scikit-learn/scikit-learn/pulls?q=is%3Apr+is%3Aclosed) (as of June 30th 2025). It is not a flood yet, but bad enough to keep several maintainers busy for some extra hours a week. It could become a flood in the future. This is why it is important to find solutions.

Quite some of the authors of these additional low-quality PRs on scikit-learn also spam llm-based PRs on other open source projects at the same time. I have added repeated cases to @adrinjalali's [agents-to-block](https://github.com/adrinjalali/agents-to-block/pull/1/files) folder. The pattern of spammers is to open a PR with an unqualified guess of what the project needs or how an issue can be solved, and then not follow up after maintainers reviewed, close and try again. 

PRs can look like someone made a genuine attempt to address an open issue, and project maintainers start to interact with the "authors" - but then their review comments are processed by an agent (not manually) and the PR never improves after a certain point.

**Many authors of automated PRs did not invest a lot of time into it and don't have enough skin in the game to care.**

This creates a huge burden for reviewers because we cannot see if a PR or issue is AI generated and there is no way to foresee how much manual work and thought their "authors" are willing to invest after their llm's first guess wasn't quite helpful.

Reviewers are forced to chose between several options:
a) being helpful to an alleged newbe who makes mistakes but wants to learn (only that they are not),
b) writing our review comments like prompts for someone's llm,
c) closing the PR by writing down a polite-to-humans closing message with a reasoning on why this PR doesn't measure up quality-wise or is not needed, even though the human author might hardly care or might not even read it 
Reviewers can loose time on trying option a and b before before closing a PR.

This takes reviewer's attention from PRs that are thoughtful which is unfair to human contributors that create thoughtful PRs and it is ultimately harmful to the whole project.

###############

The issue to solve is to find out how to protect against the effects of AI spam, mark AI based PRs and issues (so reviewers know what they deal with), most efficiently block spam-authors, and discourage LLM agents to automatically open PRs in scikit-learn.

I really think there should be a process-oriented way to deal with this problem. Ideally, we can find technical solutions.

It is not enough to let reviewers deal with on a one-to-one basis.

COMMENT:
@betatim 
I like your idea of starting with a label only and see how it goes. It is simple and will really help us prioritise review efforts. (Edit: actually, I like it as a compromise and because I want to go forward with it and this seems the something everyone who has a stake it it seems to accept. Personally, I would auto-close.)

Also the FAQ entry looks like a good draft to start with. I am thinking of also providing some context on the scarcity of reviewer time, something like:

> LLMs make it easier to generate PRs, but harder to review them, and unfortunately the scikit-learn community is lacking reviewers.

maybe also:

> Thanks for your interest, but we currently can't support low-context or partial contributions. Please see our contribution guide and open a complete, well-scoped PR if you are confident in your proposal.

(These are just some drafts.)

I however think a label  `seems ai-generated` (or similar in meaning) is a better scope, because it feels less judgy and since we are all nice people and want to be a welcoming project, I think we would use something like that more often than we would mark something as `maybe spam`.

I very much believe that many people who open llm-generated low effort PRs wouldn't see themselves as spammers and most might not be able to figure out what to do in order to improve the quality of their llm output, because you need to feed it with the right information on scikit-learn specifics that you can only know if you had spent time looking at other code in scikit-learn. Some might have even spend a lot of time trying to tweak their llm and this to them feels high-effort and not spammy.

These legit looking PRs that had been fully or partially been generated, are even the harder problem than very obvious cases, we would mark as spam and close.

A PR like #31487 maybe "only" takes a reviewer 10 minutes to conclude there is too much sloppyness and an accumulation of llm-typical mistakes that reviewing it wouldn't mean much more than talking to another person's llm.

But a PR like #31581 probably took 1 hour of reviewer time and it would have taken longer if not stopped. Or a PR like #31693, that forces a reviewer to go back and explain to contributors to check their llm output [again](https://github.com/scikit-learn/scikit-learn/pull/31693#pullrequestreview-2991901130) and [again](https://github.com/scikit-learn/scikit-learn/pull/31693#discussion_r2189568398): how many hours and nerves does that cost?

I can see that  `maybe spam` is a more general label and that has some advantages. But from experience low effort PRs are almost always generated PRs at the same time.

If that doesn't convince you (it's my last attempt, promise :crossed_fingers:), I'm happy to work on a label with another name/scope as well. Or maybe someone has a better idea for a non-judgy but clear label name?

Labels: RFC