Installing xFormers

We recommend the use of xFormers for both inference and training. In our tests, the optimizations performed in the attention blocks allow for both faster speed and reduced memory consumption.

Installing xFormers has historically been a bit involved, as binary distributions were not always up to date. Fortunately, the project has very recently integrated a process to build pip wheels as part of the project’s continuous integration, so this should improve a lot starting from xFormers version 0.0.16.

Until xFormers 0.0.16 is deployed, you can install pip wheels using TestPyPI. These are the steps that worked for us in a Linux computer to install xFormers version 0.0.15:

pip install pyre-extensions==0.0.23
pip install -i https://test.pypi.org/simple/ formers==0.0.15.dev376

We’ll update these instructions when the wheels are published to the official PyPI repository.