Create README.md

README.md

@@ -0,0 +1,48 @@
+---
+base_model: GeneZC/MiniChat-2-3B
+inference: True
+model_type: Llama
+---
+# Nous-Hermes-2-Yi-34B 
+This repo contains pruned model files for [Nous-Hermes-2-Yi-34B ](https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B).
+
+This model was pruned with [SparseGPT](https://arxiv.org/abs/2301.00774), using [SparseML](https://github.com/neuralmagic/sparseml).
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+prompt = "How to make banana bread?"
+formatted_prompt =  f"<|im_start|>User:{prompt}\n<|im_start|>assistant:\n"
+model_id = "nm-testing/Nous-Hermes-2-Yi-34B-pruned50-24"
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype=torch.float16)
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+inputs = tokenizer(formatted_prompt, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=200)
+print(tokenizer.batch_decode(outputs)[0])
+"""
+<|im_start|> User:How to make banana bread?
+<|im_start|> assistant:
+To make banana bread, you can follow these steps:
+
+Ingredients:
+- 2 ripe bananas
+- 2 cups flour
+- 1/2 cup sugar
+- 1/2 cup butter
+- 1/2 cup milk
+- 1 teaspoon baking powder
+- 1 teaspoon baking soda
+- 1 teaspoon salt
+
+Instructions:
+1. Preheat the oven to 350°F (175°C).
+2. In a mixing bowl, mash the bananas and mix them with the flour, sugar, butter, milk, baking powder, baking soda, and salt.
+3. Mix the ingredients until they form a dough.
+4. Pour the dough into a baking pan.
+5. Bake the banana bread for 30 minutes.
+6. Remove the banana bread from the oven and let it cool.
+7. Enjoy your banana bread.
+
+Note: You can adjust the ingredients
+"""
+
+```