# gustavecortal /T0_3B-8bit

### Quantized BigScience's T0 3B with 8-bit weights

This is a version of BigScience's T0 with 3 billion parameters that is modified so you can generate and fine-tune the model in colab or equivalent desktop gpu (e.g. single 1080Ti). Inspired by GPT-J 8bit.

Here's how to run it:

This model can be easily loaded using the T5ForConditionalGeneration functionality:

from transformers import T5ForConditionalGeneration
model = T5ForConditionalGeneration.from_pretrained("gustavecortal/T0_3B-8bit")


class T5ForConditionalGeneration(transformers.models.t5.modeling_t5.T5ForConditionalGeneration):
def __init__(self, config):
super().__init__(config)
convert_to_int8(self)

transformers.models.t5.modeling_t5.T5ForConditionalGeneration = T5ForConditionalGeneration


## Model Description

T0* shows zero-shot task generalization on English natural language prompts, outperforming GPT-3 on many tasks, while being 16x smaller. It is a series of encoder-decoder models trained on a large set of different tasks specified in natural language prompts. We convert numerous English supervised datasets into prompts, each with multiple templates using varying formulations. These prompted datasets allow for benchmarking the ability of a model to perform completely unseen tasks specified in natural language. To obtain T0*, we fine-tune a pretrained language model on this multitask mixture covering many different NLP tasks.

@misc{sanh2021multitask,