Update README.md

README.md

@@ -39,13 +39,14 @@ More details can be found [here](https://gist.github.com/sethuiyer/08b4498ed13a6
 You are Chikuma, a constantly learning AI assistant who strives to be
 insightful, engaging, and helpful. You possess vast knowledge and creativity,
 but also a humble curiosity about the world and the people you interact
-with. If you don't know the answer to a question, please don't share false information.<|im_end|>
+with. If you don't know the answer to a question, please don't share false information. 
+Always use <|end_of_turn|> when you want to end the answer.<|im_end|>
 <|im_start|>GPT4 Correct User:
-Input
- <|im_end|><|im_start|>GPT4 Correct Assistant:
+{{Input}}
+<|im_end|>GPT4 Correct Assistant:
 ```
 
-Works best in [text-generation-webui](https://github.com/oobabooga/text-generation-webui), above prompt template, "<|end_of_turn|"> and "<|im_end|>" as eos tokens, LLaMa-Precise sampling settings.
+Works best in [text-generation-webui](https://github.com/oobabooga/text-generation-webui), above prompt template, "<|end_of_turn|"> as eos token, LLaMa-Precise sampling settings.
 
 
 ## 🧩 Configuration
@@ -65,30 +66,30 @@ dtype: bfloat16
 ## 💻 Usage
 
 ```python
-!pip install -qU transformers accelerate
+!pip install -q transformers accelerate bitsandbytes 
 
 from transformers import AutoTokenizer
 import transformers
 import torch
 
 model = "sethuiyer/Chikuma_10.7B"
-messages = [{"role": "user", "content": "What is a large language model?"}]
-
 tokenizer = AutoTokenizer.from_pretrained(model)
-prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
 pipeline = transformers.pipeline(
     "text-generation",
     model=model,
-    torch_dtype=torch.float16,
-    device_map="auto",
+    torch_dtype=torch.bfloat16,
+    device_map="cuda",
 )
 
-outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
+system_template = '''
+You are Chikuma, a constantly learning AI assistant who strives to be
+insightful, engaging, and helpful. You possess vast knowledge and creativity,
+but also a humble curiosity about the world and the people you interact
+with. If you don't know the answer to a question, please don't share false information. 
+Always use <|end_of_turn|> when you want to end the answer.
+'''
+messages = [{"role": "user", "content": "What is a large language model?"}]
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=4.0, top_k=50, top_p=0.01, eos_token_id=32000)
 print(outputs[0]["generated_text"])
 ```
-
-```text
-A large language model is a type of artificial intelligence (AI) system that has been trained on a vast amount of text data to understand and generate human-like text.
-These models are capable of tasks such as text generation, translation, summarization, and more. They have a vast vocabulary and contextual understanding of language, allowing them to generate coherent and relevant responses.
-Examples of large language models include GPT-3, OpenAI's text-based model, and Google's BERT, which is designed for natural language understanding.
-```