You are viewing main version, which requires installation from source. If you'd like
regular pip install, checkout the latest stable version (v0.43.3).
Other algorithms
WIP: Still incomplete… Community contributions would be greatly welcome!
This is an overview of the bnb.functional
API in bitsandbytes
that we think would also be useful as standalone entities.
Using Int8 Matrix Multiplication
For straight Int8 matrix multiplication with mixed precision decomposition you can use bnb.matmul(...)
. To enable mixed precision decomposition, use the threshold parameter:
bnb.matmul(..., threshold=6.0)