Repository: Lightning-AI/pytorch-lightning (language: python)

Issue:
Drop the numpy requirement
### Outline & Motivation

We barely use `numpy`. We should replace its uses with torch.

Reduce the package footprint

### Pitch

Remove https://github.com/Lightning-AI/lightning/blob/7bbbe22636853e1ee6fa57b557b3408d45233589/requirements/pytorch/base.txt#L4

Update numpy usages with torch.

### Additional context

_No response_

cc @borda @justusschock @awaelchli


Discussion / hints:
I think there was some cross-dependency, leaking `numpy` typing which was changed in the frozen version but lets see if all is fine now :)
We had a strong dependency on numpy for working with jagged arrays when [post-processing the various output formats for the epoch_end hooks](https://github.com/Lightning-AI/lightning/blob/fc195b95405e9e2629466e5b28c6a9243209d596/src/pytorch_lightning/loops/epoch/training_epoch_loop.py#L361).
Since this was removed, we have a weaker dependency now.
One notable usage though is in the seeding: https://github.com/Lightning-AI/lightning/blob/7bbbe22636853e1ee6fa57b557b3408d45233589/src/lightning/fabric/utilities/seed.py#L96-L101

There's this issue on PyTorch to include the SeedSequence API: https://github.com/pytorch/pytorch/issues/56805, but as a workaround, they added https://github.com/pytorch/pytorch/pull/56797
The relevant discussion is at https://github.com/pytorch/pytorch/pull/56488/files#r619435608

This PR can be done in pieces, we don't need to replace all usages concurrently.
@carmocca , if this issue is planned to be done in pieces, I can help with some module. let me know. 
@dingusagar You can go ahead and replace the usages you are confident about.
@carmocca @Borda is this issue still active? I would be interested to work on it. 

Can you provide me with a more detailed context of what needs to be done? 

Do we plan to remove the numpy requirement completely and replace it with torch or this is for some particular files? 
> Do we plan to remove the numpy requirement completely and replace it with torch

This is the end goal. Some of our numpy usages are easy to replace, others [not so much](https://github.com/Lightning-AI/lightning/issues/16649#issuecomment-1419301931). You can start doing the easy replacements first, get those merged, and then addressing the hard ones separately after.

Once all are done, the requirement can be removed.
Okay, I will get started with it. @carmocca can you assign this issue to me?
@carmocca I have started off with the numpy to torch conversion, have begun with the LR Finder, and raised a PR at #17264 
I have removed the numpy usage for the callback scripts, and raised a PR #17267 
I'd also like to help on this. 

 @ishandutta0098 I can work on the model summary if you haven't yet.
Hi @ryan597 I have made changes to most of the files in lightning and they are currently on my local branch, the testing was pending so I didn't create a pull request for them yet. 

However I haven't changed anything in the examples folder yet, maybe you can give it a try. If you do kindly share the files you are modifying so that we don't change the same ones!
Sure thanks. I did a quick grep through the files and see numpy in these examples. They look like quick changes so I can do these.

examples/fabric/meta_learning/train_fabric.py
examples/fabric/meta_learning/train_torch.py
examples/fabric/reinforcement_learning/rl/agent.py
examples/fabric/reinforcement_learning/rl/utils.py

Edit to include: 
examples/fabric/reinforcement_learning/train_torch.py
examples/fabric/reinforcement_learning/train_fabric.py
examples/fabric/reinforcement_learning/train_fabric_decoupled.py
Okay, I would skip these files then.
Hey are you guys still working on this issue? Anything I could contribute?

> Hey are you guys still working on this issue? Anything I could contribute?

You can take it =) 
Hi @Peiffap, which files in `src/lightning` are you currently working at? Can I share some files with you to drop the numpy requirement. 
Thanks
Hi @Bhavay-2001, cheers for checking in with me on this!

> which files in `src/lightning` are you currently working at?

I'm not currently actively working on this issue, but I think it would make sense in general to create a [task list](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/about-task-lists) for this issue with the work left to do (i.e. the files or directories with numpy dependencies), in order to keep track of who's doing what/which tasks remain.

> Can I share some files with you to drop the numpy requirement.

I'm not sure I understand what you mean here.

Hi @Peiffap, I will make a list of files that need to be changed or updated from numpy to torch and will share. Then I can work on my half of files and you can work on yours. 
Thanks
> I will make a list of files that need to be changed or updated from numpy to torch and will share.

That sounds good!

> Then I can work on my half of files and you can work on yours.

I don't currently plan on working on this, but in any case it'd be good to have a place for people wanting to contribute to find which files are being worked on by someone else already.
There are mainly 3-4 files in the `src/lightning` folder that have the numpy library dependency. I think I can handle those myself.
Also @awaelchli, do we need to change numpy dependency in the `tests` package as well?
Hi 
Sorry for the late reply here, I've been busy with other things, but I've seen some activity here and wanted to clarify a few things.

1. The motivation of this issue is to drop the numpy requirement, but to be more precise what that means is making the numpy depencency _no longer a required dependency_. There are places in the code where we handle numpy, and these need to stay. What that means for this issue is that these places need to be guarded by a check whether numpy is available. If it is not, we do nothing. If it is available, we do what we used to do. The relevant places where this applies are:

	- [x] [Seeding the random state](https://github.com/Lightning-AI/pytorch-lightning/blob/90f23b34e956ac19eda4f5b41b9d1339595fd207/src/lightning/fabric/utilities/seed.py#L57)
	- [x] [Worker seed initializtion](https://github.com/Lightning-AI/pytorch-lightning/blob/90f23b34e956ac19eda4f5b41b9d1339595fd207/src/lightning/fabric/utilities/seed.py#L78)
	- [x] [Collection of numpy rng state](https://github.com/Lightning-AI/pytorch-lightning/blob/90f23b34e956ac19eda4f5b41b9d1339595fd207/src/lightning/fabric/utilities/seed.py#L105-L111)
	- [x] [Restoration of numpy state](https://github.com/Lightning-AI/pytorch-lightning/blob/90f23b34e956ac19eda4f5b41b9d1339595fd207/src/lightning/fabric/utilities/seed.py#L117-L124) 
	- [x] [Conversion dtypes](https://github.com/Lightning-AI/pytorch-lightning/blob/90f23b34e956ac19eda4f5b41b9d1339595fd207/src/lightning/fabric/utilities/apply_func.py#L30-L40)
	- [x] [Parameter sanitization](https://github.com/Lightning-AI/pytorch-lightning/blob/90f23b34e956ac19eda4f5b41b9d1339595fd207/src/lightning/fabric/utilities/logger.py#L133-L134)

	These are all occurrences where we don't want to (and can't) replace numpy with torch. What we would do there is simply do a check whether numpy is available, like so:

	```py
	from lightning_utilities.core.imports import RequirementCache
	
	_NUMPY_AVAILABLE = RequirementCache("numpy")
	
	if _NUMPY_AVAILABLE:
	     # execute special numpy code
	else:
	     # do nothing
	
	```
	
	**Contributions for these are welcome, please start with these if you are interested.**


2. Should we also remove the numpy from tests? The answer is no. Tests are not shipped as part of the package, and numpy is useful in our tests for various reasons. We should keep numpy as a testing requirement

3. This issue does not apply to src/lightning/app. In fact, that code will soon be removed from the repository anyway. @01AbhiSingh I've seen you've opened a PR for examples/app, but the timing for this unfortunate because we plan to remove the app code in the next day or so. Maybe you'd like to take one of the above items I've listed instead?

Thanks everyone for the willingness to contribute. I hope the directions above make it a bit clearer what should be done as the next step.


>  to be more precise what that means is making the numpy depencency _no longer a required dependency_. There are places in the code where we handle numpy, and these need to stay. What that means for this issue is that these places need to be guarded by a check whether numpy is available. If it is not, we do nothing.

So, the goal is to make the numpy an optional dependency ? Thank you for explaining it a bit more, I'll pick one of the issues you mentioned. 

Fix patch:
diff --git a/.github/workflows/_legacy-checkpoints.yml b/.github/workflows/_legacy-checkpoints.yml
index 9c11edb25271f..16072112b80a9 100644
--- a/.github/workflows/_legacy-checkpoints.yml
+++ b/.github/workflows/_legacy-checkpoints.yml
@@ -114,8 +114,8 @@ jobs:
       - name: Upload checkpoints to S3
         working-directory: ${{ env.LEGACY_FOLDER }}
         env:
-          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_PUB_ACCESS_KEY }}
-          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_PUB_SECRET_KEY }}
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_KEY }}
           AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
         run: |
           aws s3 sync $AWS_RUN checkpoints/ s3://pl-public-data/legacy/checkpoints/
diff --git a/requirements/fabric/base.txt b/requirements/fabric/base.txt
index a4ba3bc9c5757..0a99614a46870 100644
--- a/requirements/fabric/base.txt
+++ b/requirements/fabric/base.txt
@@ -1,7 +1,6 @@
 # NOTE: the upper bound for the package version is only set for CI stability, and it is dropped while installing this package
 #  in case you want to preserve/enforce restrictions on the latest compatible version, add "strict" as an in-line comment
 
-numpy >=1.21.0, <1.27.0
 torch >=2.1.0, <2.5.0
 fsspec[http] >=2022.5.0, <2024.4.0
 packaging >=20.0, <=23.1
diff --git a/requirements/pytorch/base.txt b/requirements/pytorch/base.txt
index 5aae9ae1cb468..6ff628d7edfb5 100644
--- a/requirements/pytorch/base.txt
+++ b/requirements/pytorch/base.txt
@@ -1,7 +1,6 @@
 # NOTE: the upper bound for the package version is only set for CI stability, and it is dropped while installing this package
 #  in case you want to preserve/enforce restrictions on the latest compatible version, add "strict" as an in-line comment
 
-numpy >=1.21.0, <1.27.0
 torch >=2.1.0, <2.5.0
 tqdm >=4.57.0, <4.67.0
 PyYAML >=5.4, <6.1.0
diff --git a/src/lightning/fabric/CHANGELOG.md b/src/lightning/fabric/CHANGELOG.md
index 60277c56f0a7b..aa8c23a2d2132 100644
--- a/src/lightning/fabric/CHANGELOG.md
+++ b/src/lightning/fabric/CHANGELOG.md
@@ -17,7 +17,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 - Changed the implementation of how seeds are chosen for dataloader workers when using `seed_everything(..., workers=True)` ([#20055](https://github.com/Lightning-AI/pytorch-lightning/pull/20055))
 
--
+- NumPy is no longer a required dependency ([#20090](https://github.com/Lightning-AI/pytorch-lightning/issues/20090))
 
 ### Deprecated
 
diff --git a/src/lightning/pytorch/CHANGELOG.md b/src/lightning/pytorch/CHANGELOG.md
index 2cb0d04d0aa8e..7c6fdde6e04ee 100644
--- a/src/lightning/pytorch/CHANGELOG.md
+++ b/src/lightning/pytorch/CHANGELOG.md
@@ -21,6 +21,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 - Changed the implementation of how seeds are chosen for dataloader workers when using `seed_everything(..., workers=True)` ([#20055](https://github.com/Lightning-AI/pytorch-lightning/pull/20055))
 
+- NumPy is no longer a required dependency ([#20090](https://github.com/Lightning-AI/pytorch-lightning/issues/20090))
 
 ### Deprecated
 


Test patch:
diff --git a/requirements/fabric/test.txt b/requirements/fabric/test.txt
index 3459e7dc87461..8fb9122051eec 100644
--- a/requirements/fabric/test.txt
+++ b/requirements/fabric/test.txt
@@ -1,4 +1,5 @@
 coverage ==7.3.1
+numpy >=1.17.2, <1.27.0
 pytest ==7.4.0
 pytest-cov ==4.1.0
 pytest-timeout ==2.1.0
diff --git a/requirements/pytorch/test.txt b/requirements/pytorch/test.txt
index fba9f35c7a770..4e1da300dd2fd 100644
--- a/requirements/pytorch/test.txt
+++ b/requirements/pytorch/test.txt
@@ -8,6 +8,7 @@ pytest-random-order ==1.1.0
 # needed in tests
 cloudpickle >=1.3, <2.3.0
 scikit-learn >0.22.1, <1.4.0
+numpy >=1.17.2, <1.27.0
 onnx >=1.12.0, <1.17.0
 onnxruntime >=1.12.0, <1.19.0
 psutil <5.9.6 # for `DeviceStatsMonitor`