Update README.md

README.md

@@ -14,11 +14,11 @@ license: other
 </h4>
 
 
-We opensource our **Aquila2** series, now including **Aquila2**, the base language models, namely **Aquila2-7B** and **Aquila2-34B**, as well as **AquilaChat2**, the chat models, namely **AquilaChat2-7B** and **AquilaChat2-34B**, as well as the long-text chat models, namely **AquilaChat2-7B-16k** and **AquilaChat2-34B-16k**
+We opensource our **Aquila2** series, now including **Aquila2**, the base language models, namely **Aquila2-7B**, **Aquila2-34B** and **Aquila2-70B** , as well as **AquilaChat2**, the chat models, namely **AquilaChat2-7B**, **AquilaChat2-34B** and **AquilaChat2-70B**, as well as the long-text chat models, namely **AquilaChat2-7B-16k** and **AquilaChat2-34B-16k**
 
 The additional details of the Aquila model will be presented in the official technical report. Please stay tuned for updates on official channels.
 
-## Quick Start  AquilaChat2-7B（Chat model）
+## Quick Start
 
 ### 1. Inference
 
@@ -27,29 +27,21 @@ import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
 from transformers import BitsAndBytesConfig
 
-device = torch.device("cuda:0")
-model_info = "BAAI/AquilaChat2-7B"
+model_info = "BAAI/Aquila2-70B"
 tokenizer = AutoTokenizer.from_pretrained(model_info, trust_remote_code=True)
-quantization_config=BitsAndBytesConfig(
-                        load_in_4bit=True,
-                        bnb_4bit_use_double_quant=True,
-                        bnb_4bit_quant_type="nf4",
-                        bnb_4bit_compute_dtype=torch.bfloat16,
-                    )
-model = AutoModelForCausalLM.from_pretrained(model_info, trust_remote_code=True, torch_dtype=torch.float16,
-                                                # quantization_config=quantization_config, # Uncomment this line for 4bit quantization
-                                                )
+model = AutoModelForCausalLM.from_pretrained(model_info, trust_remote_code=True)
 model.eval()
-model.to(device)
 text = "请给出10个要到北京旅游的理由。"
-from predict import predict
-out = predict(model, text, tokenizer=tokenizer, max_gen_len=200, top_p=0.95,
-              seed=1234, topk=100, temperature=0.9, sft=True, device=device,
-              model_name="AquilaChat2-7B")
-print(out)
+tokens = tokenizer.encode_plus(text)['input_ids']
+tokens = torch.tensor(tokens)[None,].to(device)
+stop_tokens = ["###", "[UNK]", "</s>"]
+with torch.no_grad():
+    out = model.generate(tokens, do_sample=True, max_length=512, eos_token_id=100007, bad_words_ids=[[tokenizer.encode(token)[0] for token in stop_tokens]])[0]
+    out = tokenizer.decode(out.cpu().numpy().tolist())
+    print(out)
 ```
 
 
 ## License
 
-Aquila2 series open-source model is licensed under [ BAAI Aquila Model Licence Agreement](https://huggingface.co/BAAI/AquilaChat2-7B/blob/main/BAAI-Aquila-Model-License%20-Agreement.pdf)
+Aquila2 series open-source model is licensed under [ BAAI Aquila Model Licence Agreement](https://huggingface.co/BAAI/Aquila2-7B/blob/main/BAAI-Aquila-Model-License-Agreement.pdf)