This is a model for testing the effect of quantization. Part of the UCSD 23Fall 240D course project. The model is a pretrained model from facebook. Model name is opt-350m. We quantize it with 8 bit quantization provided by bitsandbytes. It is a quantization technique proposed in a paper.

Downloads last month
4,153
Safetensors
Model size
331M params
Tensor type
F32
FP16
I8
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.