|
--- |
|
license: mit |
|
--- |
|
LoRA weights only and trained for research - nothing from the foundation model. |
|
Trained using Anthropics HH dataset which can be found here https://huggingface.co/datasets/Anthropic/hh-rlhf |
|
|
|
Sample usage |
|
``` |
|
import torch |
|
import os |
|
import transformers |
|
from peft import PeftModel |
|
from transformers import LlamaTokenizer, LlamaForCausalLM |
|
|
|
model_path = "decapoda-research/llama-7b-hf" |
|
peft_path = 'serpdotai/llama-hh-lora-7B' |
|
tokenizer_path = 'decapoda-research/llama-7b-hf' |
|
|
|
model = LlamaForCausalLM.from_pretrained(model_path, load_in_8bit=True, device_map="auto") # or something like {"": 0} |
|
model = PeftModel.from_pretrained(model, peft_path, torch_dtype=torch.float16, device_map="auto") # or something like {"": 0} |
|
tokenizer = LlamaTokenizer.from_pretrained(tokenizer_path) |
|
|
|
batch = tokenizer("\n\nUser: Are you sentient?\n\nAssistant:", return_tensors="pt") |
|
|
|
with torch.no_grad(): |
|
out = model.generate( |
|
input_ids=batch["input_ids"].cuda(), |
|
attention_mask=batch["attention_mask"].cuda(), |
|
max_length=100, |
|
do_sample=True, |
|
top_k=50, |
|
top_p=1.0, |
|
temperature=1.0, |
|
use_cache=False |
|
) |
|
print(tokenizer.decode(out[0])) |
|
``` |
|
|
|
The model will continue the conversation between the user and itself. If you want to use as a chatbot you can alter the generate method to include stop sequences for 'User:' and 'Assistant:' or strip off anything past the assistant's original response before returning. |
|
|
|
|
|
Trained for 2 epochs with a sequence length of 1024, mini-batch size of 3, gradient accumulation of 5, on 8 A6000s for an effective batch size of 120. |
|
|
|
Training settings: |
|
- lr: 2.0e-04 |
|
- lr_scheduler_type: linear |
|
- warmup_ratio: 0.06 |
|
- weight_decay: 0.1 |
|
- optimizer: adamw_torch_fused |
|
|
|
LoRA config: |
|
- target_modules: ['q_proj', 'k_proj', 'v_proj', 'o_proj'] |
|
- r: 64 |
|
- lora_alpha: 32 |
|
- lora_dropout: 0.05 |
|
- bias: "none" |
|
- task_type: "CAUSAL_LM" |
