Bitsandbytes

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.43.0).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

Other algorithms

WIP: Still incomplete… Community contributions would be greatly welcome!

This is an overview of the bnb.functional API in bitsandbytes that we think would also be useful as standalone entities.

Using Int8 Matrix Multiplication

For straight Int8 matrix multiplication with mixed precision decomposition you can use bnb.matmul(...). To enable mixed precision decomposition, use the threshold parameter:

bnb.matmul(..., threshold=6.0)

< > Update on GitHub

←8-bit optimizers FSDP-QLoRA→