Update README

README.md

@@ -5,21 +5,26 @@ tags:
 - llama
 ---
 
-# Model Card: OpenChat
+# OpenChat: Less is More for Open-source Models
 
-OpenChat is a series of open-source language models fine-tuned on 6K diverse and high-quality multi-round conversations. 
+OpenChat is a series of open-source language models fine-tuned on very little diverse and high-quality multi-round conversations. The [dataset](https://huggingface.co/datasets/openchat/openchat_sharegpt4_dataset) contains only ~6K GPT-4 conversations filtered from the 90K ShareGPT conversations.
 
 Generic models:
 
- - OpenChat: based on LLaMA-13B (**2048** context length)
- - OpenChat-8192: based on LLaMA-13B  (**extended to 8192** context length)
+ - OpenChat: based on LLaMA-13B (2048 context length)
+   - **105.7%** of ChatGPT score on Vicuna GPT-4 evaluation
+   - **80.87%** Win-rate on AlpacaEval
+   - **🚀 Only used 6K data for finetuning!!!**
+ - OpenChat-8192: based on LLaMA-13B  (extended to 8192 context length)
+   - **106.6%** of ChatGPT score on Vicuna GPT-4 evaluation
 
-Code models **(coming)**:
+Code models:
 
- - OpenCoder: based on StarCoder (**8192** context length)
- - OpenCoderPlus: based on StarCoderPlus (**8192** context length)
- - OpenCoderBase: based on StarCoderBase (**8192** context length)
+ - OpenCoderPlus: based on StarCoderPlus (native 8192 context length)
+   - **102.5%** of ChatGPT score on Vicuna GPT-4 evaluation
+   - **78.70%** Win-rate on AlpacaEval
 
+**NOTE:** Please load the pretrained models using *bfloat16*
 
 ## Conversation Template
 
@@ -34,15 +39,13 @@ Besides base model vocabulary, an end-of-turn token `<|end_of_turn|>` is added,
 tokenize("User:") + tokenize(user_question) + [eot_token_id] + tokenize("Assistant:")
 ```
 
-*Hint: In BPE, `tokenize(A) + tokenize(B)` not always equals to `tokenize(A + B)`*
+*Hint: In BPE, `tokenize(A) + tokenize(B)` does not always equals to `tokenize(A + B)`*
 
 Following is the code for generating the conversation templates:
 
 ```python
 @dataclass
 class ModelConfig:
-    name: str
-
     # Prompt
     system: Optional[str]
 
@@ -51,9 +54,6 @@ class ModelConfig:
     eot_token: str
     bos_token: Optional[str] = None
 
-    # Tokenize
-    max_tokens: Optional[int] = None
-
     # Get template
     def generate_conversation_template(self, tokenize_fn, tokenize_special_fn, message_list):
         tokens = []
@@ -86,19 +86,12 @@ class ModelConfig:
             else:
                 assert idx == len(message_list) - 1, "Empty message for completion must be on the last."
 
-        # Truncate to specified tokens
-        if self.max_tokens:
-            tokens = tokens[:self.max_tokens]
-            masks  = masks[:self.max_tokens]
-
         return tokens, masks
 
 
 MODEL_CONFIG_MAP = {
-    # OpenChat
+    # OpenChat / OpenChat-8192
     "openchat": ModelConfig(
-        name="OpenChat",
-
         # Prompt
         system=None,
 
@@ -109,15 +102,10 @@ MODEL_CONFIG_MAP = {
         ai_role="gpt",
         eot_token="<|end_of_turn|>",
         bos_token="<s>",
-
-        # Tokenize
-        max_tokens=2048
     ),
 
     # OpenCoder / OpenCoderPlus
     "opencoder": ModelConfig(
-        name="OpenCoder",
-
         # Prompt
         system=None,
 
@@ -128,9 +116,6 @@ MODEL_CONFIG_MAP = {
         ai_role="gpt",
         eot_token="<|end_of_turn|>",
         bos_token=None,
-
-        # Tokenize
-        max_tokens=8192
     )
 }
 ```