Adding support for low end and older GPU
Author: B0rner
Pull request: no

### Feature request

Using, training and processing models with the transformer pipeline is usually very computationally intensive.
Using this pipeline in a world with torch 1.8 or before is a difficult / impossible goal.  

The question in this feature request: is it possible to set the pipeline to work with older versions of torch, including the dependencies to other packages (accellerate 0.19,...)?
By this I also mean if it is possible to abstract the pipeline so that new models are also executable with old torch versions?

Basically the transformer pipeline can be initialized with older versions of torch. But using current models this practically does not work. Problems arise e.g. by the fact that the model cannot run completely in the RAM of the GPU with old / cheap GPUs, features like:    
`pipe.enable_model_cpu_offload()`  
`pipe.enable_xformers_memory_efficient_attention()`  
require additional packages that are not compatible with old torch versions.

However, updating the torch version with GPU support is often not possible since many older models support CUDA 10.1 at best.

### Motivation

There has been a big change at Nvidia from CUDA 10.1 to >=11.x.
11.x no longer works on many older or smaller graphics cards. Also those Geforce cards that were installed in millions laptops (GT 700m series). Since you can't easily swap these graphics cards in laptops, you're stuck with CUDA 10.1., although these devices have a compute capability that is sufficient.

If you have to use CUDA 10.1, the latest version of torch on Windows is torch 1.8 (https://download.pytorch.org/whl/cu101/torch_stable.html).

However, torch 1.8 is not compatible with xformers and is only compatible with an older version of accelerate.

As a result, these users have to calculate the pipeline on the CPU, which takes an incredibly long time and is frustrating. So they need reasonably up-to-date hardware for practical use.

This results in 2 aspects, which is why I am opening this ticket:

* sustainability: if there were any way to use older GPUs (especially in older laptops), these divices can still be taken for testing and entry into the Huggingface - world.

* Social limits: I think that whole populations are excluded when data models, pipelines, etc. only work when the latest and therefore often expensive technology is needed. It would be great if access to this kind of technology was not necessarily tied to expensive hardware, but also worked with used alternatives (in line with the aspect of sustainability).  Especially to give young people and people with limited budgets an acceptable introduction to the world of transformer pipelines.


I know that free online services are often offered as an alternative. But these services are not always available, especially hardly during the day (depending on the time zone) with GPU. This means, for example, that a school with older laptops will not be able to work well with transformer libraries either because of their equipment or because of the limitations of the free online services. Even if their equipment has a GPU that is too weak and is attached to an old torch version.

The max matrix is: CUDA 10.1 with cudnn 8.0.4 with tensorflow 2.3.4, torch 1.8 and (probably) python 3.8.


Or maybe it is possible to build torch 2.x for CUDA 10.1? There seams for example no Torch version avalibale for CUDA 10.2

### Your contribution

Unfortunately, I do not have the skills to provide solid code changes. But I'm happy to support testing with different versions, etc.

Comments:
Hey! This is very specific so most probably not planned. I hear your pain, but it's pretty much impossible to "go back in time"  in the sense that the current state of the library relies on a lot of new packages. What you should be looking for is rather `quantisation` techniques that are compatible with your device in order to need less memory, quick and easy wins with using less precision etc etc 🤗 

Hey, thanks for your feedback.
I really tried a lot to get the older technology to work, but without success. It was frustrating to be very close to the goal and then realize that Pytorch offers libraries for 1.10, which is compiled in such a way that it accepts cuda 10.2 but requires an NVIDA driver version, which is only available for new graphics cards. Then compatibility with 10.2 makes no sense, because GPUs that support the new driver can also use CUDA 12.x. Why the new cards should use CUDA 10.2?


Unfortunately, I don't know enough about this topic to fully understand your recommendations in your post. ...and benefit from it.

An option for all older devices would be a Google TPU USB stick. But the experiments in Colab do not show that various models and pipelines can be pushed into a TPU ad hoc. It's all far more complicated. Especially for a beginner like me. For example, I couldn't get a stable diffusion model to run on a Colab TPU. :-/

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.