|
--- |
|
datasets: |
|
- tatsu-lab/alpaca |
|
language: |
|
- en |
|
pipeline_tag: text2text-generation |
|
library_name: transformers |
|
license: other |
|
--- |
|
|
|
|
|
# Model Details |
|
|
|
- **Model name:** Flan-UL2-Alpaca-LORA |
|
- **Model type:** - Text2Text Generation |
|
- **Parent Model:** [google/flan-ul2](https://huggingface.co/google/flan-ul2) |
|
- **Training dataset:** [Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) |
|
- **Language:** English |
|
- **Framework:** PyTorch |
|
- **Model version:** 1.0 |
|
|
|
|
|
We take the instruction-tuned Flan models (trained on Academic datasets) and perform style transfer using the Alpaca dataset. |
|
|
|
We fine-tuned the google/flan-ul2 model on the Alpaca datset using [PEFT-LORA](https://huggingface.co/docs/diffusers/main/en/training/lora). |
|
|
|
# License |
|
- Parent model ([google/flan-ul2](https://huggingface.co/google/flan-ul2)): Apache 2.0 |
|
- Dataset ([Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca)) : cc-by-4.0 |
|
- Text-Davinci-3 (Used to generate Alpaca): [OpenAI License](https://openai.com/policies/terms-of-use) |
|
|
|
|
|
|
|
# How to Use |
|
|
|
``` |
|
import torch |
|
from transformers import pipeline |
|
|
|
# Chose the model inference precision |
|
dtype = torch.float16 # options are torch.float16, torch.bfloat16, torch.float32 |
|
|
|
model = pipeline(model="VMware/flan-ul2-alpaca-lora",device_map = 'auto',torch_dtype=dtype ) |
|
|
|
prompt = "YOUR PROMPT HERE" |
|
|
|
output = model(prompt, max_length=2048, do_sample=True) |
|
|
|
|
|
``` |
|
|
|
|
|
Using Alpaca prompt template might generate better outputs for certain prompts as the model was trained using the bellow template. |
|
|
|
``` |
|
# Chose the model inference precision |
|
import torch |
|
from transformers import pipeline |
|
|
|
dtype = torch.float16 # options are torch.bfloat16, torch.float32 |
|
model = pipeline(model="VMware/flan-ul2-alpaca-lora",device_map = 'auto',torch_dtype=dtype ) |
|
|
|
prompt_template = "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:" |
|
|
|
prompt = "YOUR PROMPT HERE" |
|
|
|
output = model(prompt_template.format(instruction= prompt), max_length=2048, do_sample=True) |
|
|
|
|
|
|
|
``` |
|
|
|
# Training Details |
|
|
|
The model was trained on 3xV100 GPUs using PEFT-LORA and Deepspeed |
|
|
|
* Hyperparameters: |
|
* learning_rate = 3e-4 |
|
* per_device_batch_size = 2 |
|
* gradient_accumulation_steps = 21 |
|
* epochs = 3 |
|
|
|
|
|
``` |
|
|
|
|
|
|
|
# Limitations and Bias |
|
|
|
The model is based on a large and diverse dataset, but it may still have limitations and biases in certain areas. Some limitations include: |
|
|
|
- Language: The model is designed to work with English text only and may not perform as well in other languages. |
|
|
|
|
|
In addition, the model may have some bias in terms of the data it was trained on. The dataset includes questions from a variety of sources, but it may not be representative of all populations or perspectives. As a result, the model may perform better or worse for certain types of questions or on certain types of texts. |
|
|
|
# Contribution |