update tokenizer + instructions

README.md

@@ -1,3 +1,14 @@
+---
+title: Alpaca LoRa 7B
+language: en
+license: other
+tags:
+- alpaca
+- lora
+- llama
+- peft
+---
+
 # Alpaca LoRa 7B
 
 This repository contains a LLaMA-7B fine-tuned model on the [Standford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) cleaned version dataset.
@@ -6,20 +17,73 @@ This repository contains a LLaMA-7B fine-tuned model on the [Standford Alpaca](h
 
 # Usage
 
+## Creating prompt
+
+The model was trained on the following kind of prompt:
+
+```python
+def generate_prompt(instruction: str, input_ctxt: str = None) -> str:
+    if input:
+        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
+
+### Instruction:
+{instruction}
+
+### Input:
+{input}
+
+### Response:"""
+    else:
+        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+{instruction}
+
+### Response:"""
+```
+
 ## Using the model
 
 ```python
-from transformers import LlamaTokenizer, LlamaForCausalLM,
+import torch
+from transformers import GenerationConfig, LlamaTokenizer, LlamaForCausalLM
 
-tokenizer = LlamaTokenizer.from_pretrained("decapoda-research/alpaca-lora-7b")
+tokenizer = LlamaTokenizer.from_pretrained("chainyo/alpaca-lora-7b")
 model = LlamaForCausalLM.from_pretrained(
     "chainyo/alpaca-lora-7b",
     load_in_8bit=True,
     torch_dtype=torch.float16,
     device_map="auto",
 )
+generation_config = GenerationConfig(
+    temperature=0.2,
+    top_p=0.75,
+    top_k=40,
+    num_beams=4,
+    max_new_tokens=128,
+)
 
 model.eval()
 if torch.__version__ >= "2":
     model = torch.compile(model)
+
+instruction = "What is the meaning of life?"
+input_ctxt = None  # For some tasks, you can provide an input context to help the model generate a better response.
+
+prompt = generate_prompt(instruction, input_ctxt)
+input_ids = tokenizer(prompt, return_tensors="pt").input_ids
+input_ids = input_ids.to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(
+        input_ids=input_ids,
+        generation_config=generation_config,
+        return_dict_in_generate=True,
+        output_scores=True,
+    )
+
+response = tokenizer.decode(outputs.sequences[0], skip_special_tokens=True)
+print(response)
+
+>>> The meaning of life is to live a life of meaning.
 ```

special_tokens_map.json

@@ -0,0 +1 @@
+{}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
+size 499723

tokenizer_config.json

@@ -0,0 +1 @@
+{"bos_token": "", "eos_token": "", "model_max_length": 1000000000000000019884624838656, "tokenizer_class": "LlaMATokenizer", "unk_token": ""}