Update README.md

README.md

@@ -1,3 +1,65 @@
 ---
 license: apache-2.0
 ---
+
+# **Meet 10.7B Solar: Elevating Performance with Upstage Depth UP Scaling!**
+
+
+# **Introduction**
+
+We introduce the first 10.7 billion (B) parameter model, [SOLAR-10.7B](https://huggingface.co/upstage/SOLAR-10.7B-v1.0). It's compact, yet remarkably powerful, and demonstrates unparalleled state-of-the-art performance in models with parameters under 30B.
+
+We developed the Depth Up-Scaling technique. Built on the Llama2 architecture, [SOLAR-10.7B](https://huggingface.co/upstage/SOLAR-10.7B-v1.0) incorporates the innovative Upstage Depth Up-Scaling. We then integrated Mistral 7B weights into the upscaled layers, and finally, continued pre-training for the entire model.
+
+Depth-Upscaled SOLAR-10.7B has remarkable performance. It outperforms models with up to 30B parameters, even surpassing the recent Mixtral 8X7B model. For detailed information, please refer to the experimental table ([link to be updated soon]).
+Solar 10.7B is an ideal choice for fine-tuning. SOLAR-10.7B offers robustness and adaptability for your fine-tuning needs. Our simple instruction fine-tuning using the SOLAR-10.7B pre-trained model yields significant performance improvements. [[link to be updated soon]]
+
+
+# **Usage Instructions**
+
+This model has been fine-tuned primarily for single-turn interactions, making it less suitable for multi-turn chat purposes.
+
+### **Version**
+
+Make sure you have the correct version of the transformers library installed:
+
+```sh
+pip install transformers==4.35.2
+```
+
+### **Loading the Model**
+
+Use the following Python code to load the model:
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("Upstage/SOLAR-10.7B-Instruct-v1.0")
+model = AutoModelForCausalLM.from_pretrained(
+    "Upstage/SOLAR-10.7B-Instruct-v1.0",
+    device_map="auto",
+    torch_dtype=torch.float16,
+)
+```
+
+### **Conducting Single-Turn Conversation**
+
+```python
+conversation = [ {'role': 'user', 'content': 'Hello?'} ] 
+
+prompt = tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True)
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device) 
+outputs = model.generate(**inputs, use_cache=True, max_length=4096) output_text = tokenizer.decode(outputs[0]) 
+print(output_text)
+```
+
+Below is an example of the output.
+```
+<s> <|im_start|>user
+Hello?<|im_end|>
+<|im_start|>assistant
+Hello, how can I assist you today?</s>
+
+```