Accelerate with `enable_model_cpu_offload()`
Whenever I run pipe.enable_model_cpu_offload(), I get that my version of accelerate should be 0.17. However, when I try to upgrade my accelerate to 0.17, I get the following
ERROR: Could not find a version that satisfies the requirement accelerate~=0.17.0 (from versions: 0.0.1, 0.1.0, 0.2.0, 0.2.1, 0.3.0, 0.4.0, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.7.0, 0.7.1, 0.8.0, 0.9.0, 0.10.0, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.13.2, 0.14.0, 0.15.0, 0.16.0) ERROR: No matching distribution found for accelerate~=0.17.0
Its not pushed to pypi because its in development.
You can clone the repo from github, which is the 0.17.0.dev0
- git clone https://github.com/huggingface/accelerate.git
- cd accelerate
- pip install .
Be warned, as though diffusers say "Almost as fast inference as when the pipeline is used without any type of offloading." I did experience significant performance loss.