Update README.md

README.md

@@ -22,7 +22,7 @@ The Mistral-7B-Instruct-v0.1 LLM is a pretrained generative text model with 7 bi
 
 ## Model Details
 
-This model was built via parameter-efficient finetuning of the [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) base model on the [jondurbin/airoboros-2.2.1](https://huggingface.co/datasets/jondurbin/airoboros-2.2.1) dataset. Finetuning was executed on 1x A100 (40 GB SXM) for roughly 3 hours.
+This model was built via parameter-efficient finetuning of the [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) base model on the [jondurbin/airoboros-2.2.1](https://huggingface.co/datasets/jondurbin/airoboros-2.2.1) dataset.
 
 - **Developed by:** Daniel Furman
 - **Model type:** Decoder-only
@@ -53,14 +53,15 @@ We use Eleuther.AI's [Language Model Evaluation Harness](https://github.com/Eleu
 <summary>Setup</summary>
 
 ```python
-!pip install -q -U transformers peft torch accelerate bitsandbytes einops sentencepiece
+!pip install -q -U transformers peft torch accelerate einops sentencepiece
+```
 
+```python
 import torch
 from peft import PeftModel, PeftConfig
 from transformers import (
     AutoModelForCausalLM,
     AutoTokenizer,
-    BitsAndBytesConfig,
 )
 ```
 
@@ -73,17 +74,14 @@ tokenizer = AutoTokenizer.from_pretrained(
     use_fast=True,
     trust_remote_code=True,
 )
-bnb_config = BitsAndBytesConfig(
-    load_in_4bit=True,
-    bnb_4bit_quant_type="nf4",
-    bnb_4bit_compute_dtype=torch.bfloat16,
-)
+
 model = AutoModelForCausalLM.from_pretrained(
     config.base_model_name_or_path,
-    quantization_config=bnb_config,
+    torch_dtype=torch.float16,
     device_map="auto",
     trust_remote_code=True,
 )
+
 model = PeftModel.from_pretrained(
     model, 
     peft_model_id
@@ -99,18 +97,31 @@ messages = [
 ]
 
 print("\n\n*** Prompt:")
-prompt = tokenizer.apply_chat_template(
+input_ids = tokenizer.apply_chat_template(
     messages,
-    tokenize=False,
-    add_generation_prompt=True
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt",
 )
-print(prompt)
+print(tokenizer.decode(input_ids[0]))
+```
+
+<details>
+
+<summary>Output</summary>
+
+**Prompt**: 
+```python
+"""
+<s> [INST] Tell me a recipe for a mai tai. [/INST]
+"""
+```
 
+```python
 print("\n\n*** Generate:")
-input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()
 with torch.autocast("cuda", dtype=torch.bfloat16):
     output = model.generate(
-        input_ids=input_ids,
+        input_ids=input_ids.cuda(),
         max_new_tokens=1024,
         do_sample=True,
         temperature=0.7,
@@ -126,20 +137,21 @@ response = tokenizer.decode(
     skip_special_tokens=True
 )
 print(response)
-```
-
-<details>
 
-<summary>Output</summary>
-
-**Prompt**: 
-```python
-coming
 ```
 
 **Generation**:
 ```python
-coming
+"""1 oz light rum
+½ oz dark rum
+¼ oz orange curaçao
+2 oz pineapple juice
+¾ oz lime juice
+Dash of orgeat syrup (optional)
+Splash of grenadine (for garnish, optional)
+Lime wheel and cherry garnishes (optional)
+
+Shake all ingredients except the splash of grenadine in a cocktail shaker over ice. Strain into an old-fashioned glass filled with fresh ice cubes. Gently pour the splash of grenadine down the side of the glass so that it sinks to the bottom. Add garnishes as desired."""
 ```
 
 </details>
@@ -153,7 +165,7 @@ coming
 
 ## Training
 
-It took ~3 hours to train 3 epochs on 1x A100 (40 GB SXM).
+It took ~2 hours to train 2 epochs on 1x A100 (40 GB SXM).
 
 ### Prompt Format
 
@@ -168,12 +180,20 @@ This format is available as a [chat template](https://huggingface.co/docs/transf
 
 ```python
 messages = [
-    {"role": "user", "content": "What is your favourite condiment?"},
-    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
-    {"role": "user", "content": "Do you have mayonnaise recipes?"}
+    {"role": "user", "content": "Tell me a recipe for a mai tai."},
+    {"role": "assistant", "content": "1 oz light rum\n½ oz dark rum\n¼ oz orange curaçao\n2 oz pineapple juice\n¾ oz lime juice\nDash of orgeat syrup (optional)\nSplash of grenadine (for garnish, optional)\nLime wheel and cherry garnishes (optional)\n\nShake all ingredients except the splash of grenadine in a cocktail shaker over ice. Strain into an old-fashioned glass filled with fresh ice cubes. Gently pour the splash of grenadine down the side of the glass so that it sinks to the bottom. Add garnishes as desired."},
+    {"role": "user", "content": "How can I make it more upscale and luxurious?"},
+
 ]
-prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-print(prompt)
+
+print("\n\n*** Prompt:")
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt",
+)
+print(tokenizer.decode(input_ids[0]))
 ```
 
 <details>
@@ -181,7 +201,7 @@ print(prompt)
 <summary>Output</summary>
 
 ```python
-coming
+<s> [INST] Tell me a recipe for a mai tai. [/INST] 1 oz light rum\n½ oz dark rum\n¼ oz orange curaçao\n2 oz pineapple juice\n¾ oz lime juice\nDash of orgeat syrup (optional)\nSplash of grenadine (for garnish, optional)\nLime wheel and cherry garnishes (optional)\n\nShake all ingredients except the splash of grenadine in a cocktail shaker over ice. Strain into an old-fashioned glass filled with fresh ice cubes. Gently pour the splash of grenadine down the side of the glass so that it sinks to the bottom. Add garnishes as desired.</s>  [INST] How can I make it more upscale and luxurious? [/INST]
 ```
 </details>