Update README.md

README.md

@@ -1,3 +1,56 @@
 ---
 license: apache-2.0
 ---
+
+# **Meet 10.7B Solar: Elevating Performance with Upstage Depth UP Scaling!**
+
+
+# **Introduction**
+
+We introduce the first 10.7 billion (B) parameter model, SOLAR-10.7B. It's compact, yet remarkably powerful, and demonstrates unparalleled state-of-the-art performance in models with parameters under 30B.
+
+We developed the Depth Up-Scaling technique. Built on the Llama2 architecture, SOLAR-10.7B incorporates the innovative Upstage Depth Up-Scaling. We then integrated Mistral 7B weights into the upscaled layers, and finally, continued pre-training for the entire model.
+
+Depth-Upscaled SOLAR-10.7B has remarkable performance. It outperforms models with up to 30B parameters, even surpassing the recent Mixtral 8X7B model. For detailed information, please refer to the experimental table ([link to be updated soon]).
+Solar 10.7B is an ideal choice for fine-tuning. SOLAR-10.7B offers robustness and adaptability for your fine-tuning needs. Our simple instruction fine-tuning using the SOLAR-10.7B pre-trained model yields significant performance improvements. [[link to be updated soon]]
+
+
+# **Usage Instructions**
+
+This model is pre-trained and is capable of just generating random text. To use it for chatting, you must fine-tune the model first.
+
+### **Version**
+
+Make sure you have the correct version of the transformers library installed:
+
+```sh
+pip install transformers==4.35.2
+```
+
+### **Loading the Model**
+
+Use the following Python code to load the model:
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("Upstage/SOLAR-10.7B-v1.0")
+model = AutoModelForCausalLM.from_pretrained(
+    "Upstage/SOLAR-10.7B-v1.0",
+    device_map="auto",
+    torch_dtype=torch.float16,
+)
+```
+
+### **Generating Text**
+
+To generate text, use the following Python code:
+
+```python
+text = "Hi, my name is "
+inputs = tokenizer(text, return_tensors="pt")
+
+outputs = model.generate(**inputs, max_new_tokens=64)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```