Upload README.md

README.md

@@ -1,3 +1,54 @@
 ---
-license: cc-by-sa-4.0
+language:
+- en
+tags:
+- causal-lm
+license: cc-by-nc-sa-4.0
+datasets:
+- dmayhem93/ChatCombined
+- tatsu-lab/alpaca
+- nomic-ai/gpt4all_prompt_generations
+- Dahoas/full-hh-rlhf
+- jeffwan/sharegpt_vicuna
+- HuggingFaceH4/databricks_dolly_15k
 ---
+
+# StableLM-Tuned-Alpha 16-bit
+
+## Model Description
+
+16-bit version of `StableLM-Tuned-Alpha` compressed for the sake of speed and memory usage. No other changes were made. Original model: https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b
+
+## Usage
+
+Get started chatting with `StableLM-Tuned-Alpha 16-bit` by using the following code snippet:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList
+tokenizer = AutoTokenizer.from_pretrained("vvsotnikov/stablelm-tuned-alpha-7b-16bit")
+model = AutoModelForCausalLM.from_pretrained("vvsotnikov/stablelm-tuned-alpha-7b-16bit")
+model.cuda()
+class StopOnTokens(StoppingCriteria):
+    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
+        stop_ids = [50278, 50279, 50277, 1, 0]
+        for stop_id in stop_ids:
+            if input_ids[0][-1] == stop_id:
+                return True
+        return False
+system_prompt = """<|SYSTEM|># StableLM Tuned (Alpha version)
+- StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.
+- StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
+- StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.
+- StableLM will refuse to participate in anything that could harm a human.
+"""
+prompt = f"{system_prompt}<|USER|>What's your mood today?<|ASSISTANT|>"
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+tokens = model.generate(
+  **inputs,
+  max_new_tokens=64,
+  temperature=0.7,
+  do_sample=True,
+  stopping_criteria=StoppingCriteriaList([StopOnTokens()])
+)
+print(tokenizer.decode(tokens[0], skip_special_tokens=True))
+```