Update README.md

README.md

@@ -8,4 +8,38 @@ This repo contains pruned model files for [Nous-Hermes-2-SOLAR-10.7B](https://hu
 
 This model was pruned with [SparseGPT](https://arxiv.org/abs/2301.00774), using [SparseML](https://github.com/neuralmagic/sparseml).
 ```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+prompt = "How to make banana bread?"
+formatted_prompt =  f"### User:\n{prompt}\n\n### Assistant:\n"
+model_id = "nm-testing/Nous-Hermes-2-SOLAR-10.7B-pruned50-24"
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype=torch.float16)
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+inputs = tokenizer(formatted_prompt, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=200)
+print(tokenizer.batch_decode(outputs)[0])
+"""
+<s> ### User:
+How to make banana bread?
+
+### Assistant:
+To make banana bread, you will need the following ingredients:
+
+- 1 cup of flour
+- 1 cup of sugar
+- 3 cups of bananas
+- 1 cup of milk
+- 1 cup of eggs
+- 1 cup of baking powder
+
+Instructions:
+
+1. Mix the flour and sugar together.
+2. Add the bananas to the mixture.
+3. Add the milk and eggs to the mixture.
+4. Add the baking powder to the mixture.
+5. Mix all the ingredients together.
+6. Pour the mixture into a baking pan.
+7. Bake the mixture for 30 minutes at 350 degrees.<|im_end|>
+"""
 ```