Edit model card
pomeranian logo

PomeranIAn

Adapter Description

This adapter was created with the PEFT library and allowed the base model Falcon-7b to be fine-tuned on the Spanish instructions dataset by using the method QLoRA.

Model description

Falcon 7B

Intended uses & limitations

TBA

Training and evaluation data

TBA

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 8

Training results

Training Loss Epoch Step Validation Loss
1.6302 1.54 10 1.6542
1.5742 3.08 20 1.6157
1.3779 4.62 30 1.5896
1.2988 6.15 40 1.5753

How to use

import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, AutoTokenizer, GenerationConfig

peft_model_id = "mrm8488/pomeranian"

config = PeftConfig.from_pretrained(peft_model_id)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
)

tokenizer = AutoTokenizer.from_pretrained(peft_model_id)

model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path,
                                             return_dict=True,
                                             quantization_config=bnb_config, 
                                             trust_remote_code=True, 
                                             device_map={"":0})


prompt_input = "A continuaci贸n, se muestra una instrucci贸n que describe una tarea, junto con una entrada que proporciona m谩s contexto. Escribe una respuesta que complete adecuadamente la solicitud.\n\n### Instrucci贸n:\n{instruction}\n\n### Entrada:\n{input}\n\n### Respuesta:\n"
prompt_no_input = "A continuaci贸n, se muestra una instrucci贸n que describe una tarea. Escribe una respuesta que complete adecuadamente la solicitud.\n\n### Instrucci贸n:\n{instruction}\n\n### Respuesta:\n"

def create_prompt(instruction, input=None):
  if input:
    return  prompt_input.format(instruction=instruction, input=input)
  else:
    return prompt_no_input.format(instruction=instruction)

def generate(
        instruction,
        input=None,
        max_new_tokens=128,
        temperature=0.1,
        top_p=0.75,
        top_k=40,
        num_beams=4,
        **kwargs,
):
    prompt = create_prompt(instruction, input)
    inputs = tokenizer(prompt, return_tensors="pt")
    input_ids = inputs["input_ids"].to("cuda")
    attention_mask = inputs["attention_mask"].to("cuda")
    generation_config = GenerationConfig(
        temperature=temperature,
        top_p=top_p,
        top_k=top_k,
        num_beams=num_beams,
        **kwargs,
    )
    with torch.no_grad():
        generation_output = model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,
            generation_config=generation_config,
            return_dict_in_generate=True,
            output_scores=True,
            max_new_tokens=max_new_tokens
        )
    s = generation_output.sequences[0]
    output = tokenizer.decode(s)
    return output.split("### Respuesta:")[1]

instruction = "Dime algo sobre los halcones"

print("Instrucci贸n:", instruction)
print("Respuesta:", generate(instruction))

Framework versions

  • Transformers 4.30.0.dev0
  • Pytorch 2.0.1+cu118
  • Datasets 2.12.0
  • Tokenizers 0.13.3
Downloads last month
0
Unable to determine this model's library. Check the docs .