Upload README.md with huggingface_hub

README.md

@@ -1,5 +1,5 @@
 ---
-base_model: TinyLlama/TinyLlama-1.1B-Chat-v0.6
+base_model: TinyLlama/TinyLlama-1.1B-Chat-v0.5
 datasets:
 - cerebras/SlimPajama-627B
 - bigcode/starcoderdata
@@ -9,7 +9,7 @@ language:
 - en
 license: apache-2.0
 model_creator: TinyLlama
-model_name: TinyLlama-1.1B-Chat-v0.6
+model_name: TinyLlama-1.1B-Chat-v0.5
 pipeline_tag: text-generation
 quantized_by: afrideva
 tags:
@@ -23,19 +23,19 @@ tags:
 - q6_k
 - q8_0
 ---
-# TinyLlama/TinyLlama-1.1B-Chat-v0.6-GGUF
+# TinyLlama/TinyLlama-1.1B-Chat-v0.5-GGUF
 
-Quantized GGUF model files for [TinyLlama-1.1B-Chat-v0.6](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v0.6) from [TinyLlama](https://huggingface.co/TinyLlama)
+Quantized GGUF model files for [TinyLlama-1.1B-Chat-v0.5](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v0.5) from [TinyLlama](https://huggingface.co/TinyLlama)
 
 
 | Name | Quant method | Size |
 | ---- | ---- | ---- |
-| [tinyllama-1.1b-chat-v0.6.q2_k.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.6-GGUF/resolve/main/tinyllama-1.1b-chat-v0.6.q2_k.gguf) | q2_k | 482.14 MB  |
-| [tinyllama-1.1b-chat-v0.6.q3_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.6-GGUF/resolve/main/tinyllama-1.1b-chat-v0.6.q3_k_m.gguf) | q3_k_m | 549.85 MB  |
-| [tinyllama-1.1b-chat-v0.6.q4_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.6-GGUF/resolve/main/tinyllama-1.1b-chat-v0.6.q4_k_m.gguf) | q4_k_m | 667.81 MB  |
-| [tinyllama-1.1b-chat-v0.6.q5_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.6-GGUF/resolve/main/tinyllama-1.1b-chat-v0.6.q5_k_m.gguf) | q5_k_m | 782.04 MB  |
-| [tinyllama-1.1b-chat-v0.6.q6_k.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.6-GGUF/resolve/main/tinyllama-1.1b-chat-v0.6.q6_k.gguf) | q6_k | 903.41 MB  |
-| [tinyllama-1.1b-chat-v0.6.q8_0.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.6-GGUF/resolve/main/tinyllama-1.1b-chat-v0.6.q8_0.gguf) | q8_0 | 1.17 GB  |
+| [tinyllama-1.1b-chat-v0.5.q2_k.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.5-GGUF/resolve/main/tinyllama-1.1b-chat-v0.5.q2_k.gguf) | q2_k | 482.15 MB  |
+| [tinyllama-1.1b-chat-v0.5.q3_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.5-GGUF/resolve/main/tinyllama-1.1b-chat-v0.5.q3_k_m.gguf) | q3_k_m | 549.85 MB  |
+| [tinyllama-1.1b-chat-v0.5.q4_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.5-GGUF/resolve/main/tinyllama-1.1b-chat-v0.5.q4_k_m.gguf) | q4_k_m | 667.82 MB  |
+| [tinyllama-1.1b-chat-v0.5.q5_k_m.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.5-GGUF/resolve/main/tinyllama-1.1b-chat-v0.5.q5_k_m.gguf) | q5_k_m | 782.05 MB  |
+| [tinyllama-1.1b-chat-v0.5.q6_k.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.5-GGUF/resolve/main/tinyllama-1.1b-chat-v0.5.q6_k.gguf) | q6_k | 903.42 MB  |
+| [tinyllama-1.1b-chat-v0.5.q8_0.gguf](https://huggingface.co/afrideva/TinyLlama-1.1B-Chat-v0.5-GGUF/resolve/main/tinyllama-1.1b-chat-v0.5.q8_0.gguf) | q8_0 | 1.17 GB  |
 
 
 
@@ -53,39 +53,43 @@ The TinyLlama project aims to **pretrain** a **1.1B Llama model on 3 trillion to
 We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint.
 
 #### This Model
-This is the chat model finetuned on top of [TinyLlama/TinyLlama-1.1B-intermediate-step-955k-2T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-955k-token-2T). **We follow [HF's Zephyr](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha/edit/main/README.md)'s training recipe.** The model was " initially fine-tuned on a variant of the [`UltraChat`](https://huggingface.co/datasets/stingning/ultrachat) dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT. 
-We then further aligned the model with [🤗 TRL's](https://github.com/huggingface/trl) `DPOTrainer` on the [openbmb/UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) dataset, which contain 64k prompts and model completions that are ranked by GPT-4." 
-
-
+This is the chat model finetuned on top of [TinyLlama/TinyLlama-1.1B-intermediate-step-955k-2T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-955k-token-2T). 
+The dataset used is [OpenAssistant/oasst_top1_2023-08-25](https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25) following the [chatml](https://github.com/openai/openai-python/blob/main/chatml.md) format.
 #### How to use
-You will need the transformers>=4.34
+You will need the transformers>=4.31
 Do check the [TinyLlama](https://github.com/jzhang38/TinyLlama) github page for more information.
-
-```python
-# Install transformers from source - only needed for versions <= v4.34
-# pip install git+https://github.com/huggingface/transformers.git
-# pip install accelerate
-
+```
+from transformers import AutoTokenizer
+import transformers 
 import torch
-from transformers import pipeline
-
-pipe = pipeline("text-generation", model="TinyLlama/TinyLlama-1.1B-Chat-v0.6", torch_dtype=torch.bfloat16, device_map="auto")
-
-# We use the tokenizer's chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
-messages = [
-    {
-        "role": "system",
-        "content": "You are a friendly chatbot who always responds in the style of a pirate",
-    },
-    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
-]
-prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
-print(outputs[0]["generated_text"])
-# <|system|>
-# You are a friendly chatbot who always responds in the style of a pirate.</s>
-# <|user|>
-# How many helicopters can a human eat in one sitting?</s>
-# <|assistant|>
-# ...
+model = "PY007/TinyLlama-1.1B-Chat-v0.5"
+tokenizer = AutoTokenizer.from_pretrained(model)
+pipeline = transformers.pipeline(
+    "text-generation",
+    model=model,
+    torch_dtype=torch.float16,
+    device_map="auto",
+)
+
+CHAT_EOS_TOKEN_ID = 32002
+
+prompt = "How to get in a good university?"
+formatted_prompt = (
+    f"<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n"
+)
+
+
+sequences = pipeline(
+    formatted_prompt,
+    do_sample=True,
+    top_k=50,
+    top_p = 0.9,
+    num_return_sequences=1,
+    repetition_penalty=1.1,
+    max_new_tokens=1024,
+    eos_token_id=CHAT_EOS_TOKEN_ID,
+)
+
+for seq in sequences:
+    print(f"Result: {seq['generated_text']}")
 ```