Talip7/scikit-learn-issues-multilabel huggingface dataset:
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
> An environment for which there are no wheels is not related to our dependencies, right ? It only depends on the os and python version. So I don't see how setting an upper bound on our dependencies would solve this issue.

For that we'd stop shipping the sdist.

> I can see that a new version of a dependency can break an old version of sklearn, but usually our answer in this situation is to tell to use a more recent version of scikit-learn. I think not setting upper bound helps sklearn users moving forward and not stay stuck with an old version, and I'm afraid that setting upper bounds would give an opportunity for many users to pin a specific version of sklearn forever.

Users install older versions for all sorts of reasons, including to be able to load a model persisted with an older version (they need to install exactly the same version of sklearn not to get a warning).

> To avoid this kind of thing we would need to have done upper-bound pinning from the start ...

That's true, but a few years down the line, then these very old versions become somewhat less relevant.

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
> Users install older versions for all sorts of reasons, including to be able to load a model persisted with an older version (they need to install exactly the same version of sklearn not to get a warning).

For this use case, I would say that they also need to store the version of all their other dependencies, otherwise they will avoid the warning (great for them) but may still get quite different results.

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
I've heard arguments from both sides for including upper bounds. Currently, I am -1 on having upper bounds for dependencies for a library like scikit-learn unless we are sure that it does not work with a specific version. In general, providing a upper bound gives less control to the user. If scikit-learn 1.3 does not work with a newer version of SciPy, the user can downgrade. If we cap the SciPy version, then there is nothing they can do if their environment can not be resolved because of the additional constraint.

That being said, I am +0.5 on capping the Python version on the release branch. For the Python version, we are sure that new Python versions will have a bad experience because we did not build wheels for them.

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
I am -1 on setting upper version bounds on scikit-learn's dependencies.

An application should manage the versions of its dependencies (by pinning, or what not) to achieve reproducible deploys. However, scikit-learn is a library, used by applications. As creator of an application I want as much flexibility as possible when it comes to combining different libraries. scikit-learn can't know in what environments with which other libraries it might get deployed and whether or not it is compatible with them.

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
> That being said, I am +0.5 on capping the Python version on the release branch. For the Python version, we are sure that new Python versions will have a bad experience because we did not build wheels for them.

With "bad experience" you mean that people will end up trying to compile it from source, which is likely to fail?

Maybe the solution to this is to sync our "summer" releases with the release of new minor Python versions. I think they tend to happen around the end of September/start of October.

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
I am also -1, as I see this putting hard constraints on users that are artificial. People end up in all kind of fairly constraint settings, for instance having to work jointly with many packages, so non minor.

For instance, my twitter timeline this morning was discussing a package wrote 10 years ago and no longer maintained to do specific multivariate polynomials useful in combinatorial algebra:
https://twitter.com/PyViv/status/1682058571008348161

The logic of always moving along with tech is adapted to large revenue-generating situations, but not to the long tail of less central applications.

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
> With "bad experience" you mean that people will end up trying to compile it from source, which is likely to fail?

Yup.

> Maybe the solution to this is to sync our "summer" releases with the release of new minor Python versions. I think they tend to happen around the end of September/start of October.

I think we are okay in this front. Historically, we update the current scikit-learn release with a bug fix to support the current Python release.

Concretely, I am thinking about those who are trying to scikit-learn 1.1 on Python 3.15. We are no longer updating scikit-learn 1.1, so should 1.1 have a upper bound on Python version? Thinking about it more, I am -0.5 on bounding Python versions. Usually, we recommend downgrading to a Python version that has a wheel.

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
Ah ok. I think it makes sense to use the same logic for Python versions as for other dependencies. In particular because scikit-learn 1.1 isn't strictly incompatible with Python 3.15. It is "only" missing wheels. It is unfortunate that we can't better help unsuspecting users, but we shouldn't stand in the way of expert users who want to do this.

Maybe something to add to https://scikit-learn.org/stable/faq.html?

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
I‘m also -1 on setting upper bounds on dependencies with similar concerns as already mentioned and som bad experience with an upper bound on a python version.

Given the feedback so far, can we close this RFC or give it some more time (and tag more people)?

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
Ok, I guess we can close.

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
We really are getting a bunch of traffic because of this though, another example: https://github.com/scikit-learn/scikit-learn/discussions/26910

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
Thanks for pointing this out, Adrin.

Is the issue here an incompatibility with a later released version of numpy?

If so,  would the issue be solved by an upper bound added retrospectively? Maybe something we could do in a "post" release?

On Jul 27, 2023, 08:55, at 08:55, Adrin Jalali ***@***.***> wrote:
>We really are getting a bunch of traffic because of this though,
>another example:
>https://github.com/scikit-learn/scikit-learn/discussions/26910
>
>-- 
>Reply to this email directly or view it on GitHub:
>https://github.com/scikit-learn/scikit-learn/issues/26869#issuecomment-1653015653
>You are receiving this because you are on a team that was mentioned.
>
>Message ID:
>***@***.***>

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
In this last case, the issue is a new released Cython I think, but in other cases it's not having wheels for new Python versions.

I did think of the `post` release too, but it doesn't really fix the issue we have. @lesteve is right that we would have needed to do this from the start. We recommend users pin their `scikit-learn` version to be able to load old pickle files, and for that purpose, the `post` release will help them to have a working env, but it doesn't help them with the warning that they're loading in a version which doesn't match the one they used to persist their model (we could change the error message to ignore post releases maybe though, since the warning is void if the difference is just being a `post` release)

Labels: RFC
RFC Cap uppper versions of dependencies

We've been seeing quite a few issues where people try to install scikit-learn in an environment for which there are no wheels and therefore pip trying to build the package from the source distribution, but failing.

Making scikit-learn work with new versions of dependencies, including python is not trivial, and there are no guarantees for the package to work correctly even if `pip` somehow magically manages to build from the sdist.

I propose we set the higher limit of our dependencies to whatever's released at the time of the release, so that when somebody tries to do `pip install scikit-learn==1.4` in 2-3 years from now, it either fails or downgrades the dependencies to the right ones. The same thing for supporting versions of python.

We can also stop putting sdist out there. If the users want to build the project, they probably can build it from github.

cc @scikit-learn/core-devs

COMMENT:
There is not enough information in https://github.com/scikit-learn/scikit-learn/discussions/26910 to debug the issue. I suspect they are installing an older version of scikit-learn, because `1.3.0` should be compatible with Cython 3.0.

> In this last case, the issue is a new released Cython I think, but in other cases it's not having wheels for new Python versions.

For build dependencies such as Cython, I think it's okay to cap them or even pin them. For example, our NumPy build dependency uses `oldest-supported-numpy` which [pins](https://github.com/scipy/oldest-supported-numpy/blob/main/setup.cfg) the NumPy version.

Labels: RFC
DOC the pipeline user guide should include same estimators to demonstrate pipeline construction

### Describe the issue linked to the documentation

In the  [user guide for pipelines](https://scikit-learn.org/dev/modules/compose.html) the demonstration of the construction of a pipeline includes two examples:

1. Directly creating a new instance of `Pipeline`.
2. Using the utility function `make_pipeline`.

Each example uses different lists of estimators, whereas it would be clearer if both examples used the same estimators.

### Suggest a potential alternative/fix

Both examples should use the same estimators when creating a Pipeline, namely the combination of PCA and SVC as in the first example.

In the file doc/modules/compose.rst lines 68-72 should be replaced with the following
`>>> make_pipeline(PCA(), SVC())`
`Pipeline(steps=[('reduce_dim', PCA()), ('clf', SVC())])`

COMMENT:
I'd support this change.

From a quick browse of the docs I can't quickly see why two different lists were chosen. Maybe there is a reason? But if we can't remember, it would make it easier to understand the two options if they did the same thing (but in different ways).

Labels: Documentation
DOC the pipeline user guide should include same estimators to demonstrate pipeline construction

### Describe the issue linked to the documentation

In the  [user guide for pipelines](https://scikit-learn.org/dev/modules/compose.html) the demonstration of the construction of a pipeline includes two examples:

1. Directly creating a new instance of `Pipeline`.
2. Using the utility function `make_pipeline`.

Each example uses different lists of estimators, whereas it would be clearer if both examples used the same estimators.

### Suggest a potential alternative/fix

Both examples should use the same estimators when creating a Pipeline, namely the combination of PCA and SVC as in the first example.

In the file doc/modules/compose.rst lines 68-72 should be replaced with the following
`>>> make_pipeline(PCA(), SVC())`
`Pipeline(steps=[('reduce_dim', PCA()), ('clf', SVC())])`

COMMENT:
@noashin feel free to open a PR with the fix.

Labels: Documentation
DOC the pipeline user guide should include same estimators to demonstrate pipeline construction

### Describe the issue linked to the documentation

In the  [user guide for pipelines](https://scikit-learn.org/dev/modules/compose.html) the demonstration of the construction of a pipeline includes two examples:

1. Directly creating a new instance of `Pipeline`.
2. Using the utility function `make_pipeline`.

Each example uses different lists of estimators, whereas it would be clearer if both examples used the same estimators.

### Suggest a potential alternative/fix

Both examples should use the same estimators when creating a Pipeline, namely the combination of PCA and SVC as in the first example.

In the file doc/modules/compose.rst lines 68-72 should be replaced with the following
`>>> make_pipeline(PCA(), SVC())`
`Pipeline(steps=[('reduce_dim', PCA()), ('clf', SVC())])`

COMMENT:
I prefer to use `Pipeline` for construction if it is going into a `*SearchCV`. This way I get to choose the names, which makes it easier to define the hyper-parameters in the search space.

Labels: Documentation