Update README.md

README.md

@@ -29,46 +29,53 @@ pipeline_tag: text-generation
 ```
 ### System:
 {System}
+
 ### User:
 {User}
+
 ### Assistant:
 {Assistant}
 ```
-### Usage
+
+## Usage
+
 - Tested on A100 80GB
-- Our model can handle under 10k input tokens thanks to the `rope_scaling` option
+- Our model can handle up to 10k input tokens, thanks to the `rope_scaling` option
 
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
+
 tokenizer = AutoTokenizer.from_pretrained("upstage/Llama-2-70b-instruct-v2")
 model = AutoModelForCausalLM.from_pretrained(
     "upstage/Llama-2-70b-instruct-v2",
-    device_map='auto',
+    device_map="auto",
     torch_dtype=torch.float16,
     load_in_8bit=True,
-    rope_scaling={'type': 'dynamic', 'factor': 2} # longer inputs possible
+    rope_scaling={"type": "dynamic", "factor": 2} # allows handling of longer inputs
 )
-prompt = "### User:\nThomas is very healthy, but he has to go to the hospital every day. What could be the reasons?\n\n### Assistant:\n"
+
+prompt = "### User:\nThomas is healthy, but he has to go to the hospital. What could be the reasons?\n\n### Assistant:\n"
 inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
-del inputs['token_type_ids']
+del inputs["token_type_ids"]
 streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
+
 output = model.generate(**inputs, streamer=streamer, use_cache=True, max_new_tokens=float('inf'))
 output_text = tokenizer.decode(output[0], skip_special_tokens=True)
 ```
 
-
 ## Hardware and Software
 
 * **Hardware**: We utilized an A100x8 * 4 for training our model
-* **Training Factors**: We fine-tuned this model using a combination of the [DeepSpeed library](https://github.com/microsoft/DeepSpeed) and the [HuggingFace trainer](https://huggingface.co/docs/transformers/main_classes/trainer) / [HuggingFace Accelerate](https://huggingface.co/docs/accelerate/index)
+* **Training Factors**: We fine-tuned this model using a combination of the [DeepSpeed library](https://github.com/microsoft/DeepSpeed) and the [HuggingFace Trainer](https://huggingface.co/docs/transformers/main_classes/trainer) / [HuggingFace Accelerate](https://huggingface.co/docs/accelerate/index)
 
 ## Evaluation Results
 
 ### Overview
 - We conducted a performance evaluation based on the tasks being evaluated on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
-We evaluated our model on four benchmark datasets, which include `ARC-Challenge`, `HellaSwag`, `MMLU`, and `TruthfulQA`.
+We evaluated our model on four benchmark datasets, which include `ARC-Challenge`, `HellaSwag`, `MMLU`, and `TruthfulQA`
 We used the [lm-evaluation-harness repository](https://github.com/EleutherAI/lm-evaluation-harness), specifically commit [b281b0921b636bc36ad05c0b0b0763bd6dd43463](https://github.com/EleutherAI/lm-evaluation-harness/tree/b281b0921b636bc36ad05c0b0b0763bd6dd43463).
+- We used [MT-bench](https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge), a set of challenging multi-turn open-ended questions, to evaluate the models
 
 ### Main Results
 | Model | H4(Avg) | ARC | HellaSwag | MMLU | TruthfulQA | | MT_Bench |
@@ -82,7 +89,7 @@ We used the [lm-evaluation-harness repository](https://github.com/EleutherAI/lm-
 | llama-65b | 64.2 | 63.5 | 86.1 | 63.9 | 43.4 | | |
 | falcon-40b-instruct | 63.4 | 61.6 | 84.3 | 55.4 | 52.5 | | |
 
-### Scripts
+### Scripts for H4 Score Reproduction
 - Prepare evaluation environments:
 ```
 # clone the repository
@@ -96,9 +103,9 @@ cd lm-evaluation-harness
 ## Ethical Issues
 
 ### Ethical Considerations
-- There were no ethical issues involved, as we did not include the benchmark test set or the training set in the model's training process.
+- There were no ethical issues involved, as we did not include the benchmark test set or the training set in the model's training process
 
 ## Contact Us
 
 ### Why Upstage LLM?
-- [Upstage](https://en.upstage.ai)'s LLM research has yielded remarkable results. Our 70B model **outperforms all models around the world**,  positioning itself as the leading performer. Recognizing the immense potential in implementing private LLM to actual businesses, we invite you to easily apply private LLM and fine-tune it with your own data. For a seamless and tailored solution, please do not hesitate to reach out to us. ► [click here to contact](https://www.upstage.ai/private-llm?utm_source=huggingface&utm_medium=link&utm_campaign=privatellm).
+- [Upstage](https://en.upstage.ai)'s LLM research has yielded remarkable results. Our 70B model **outperforms all models around the world**,  positioning itself as the leading performer. Recognizing the immense potential in implementing private LLM to actual businesses, we invite you to easily apply private LLM and fine-tune it with your own data. For a seamless and tailored solution, please do not hesitate to reach out to us. ► [click here to contact](https://www.upstage.ai/private-llm?utm_source=huggingface&utm_medium=link&utm_campaign=privatellm)