README.md

metadata

language:
  - zh
pipeline_tag: text-generation

FP16 Model converted from AquilaChat-7b v0.6 Pytorch Model: https://github.com/FlagAI-Open/FlagAI/tree/master/examples/Aquila

Support Inference with AutoModelForCausalLM, ORTModelForCausalLM and OVModelForCausalLM

#!pip install transformers>=4.29.2
#!pip install optimum>=1.8.7 optimum-intel[openvino]==1.9.1
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained('sammysun0711/aquilachat-7b-hf')
model = AutoModelForCausalLM.from_pretrained('sammysun0711/aquilachat-7b-hf', trust_remote_code=True)
model = model.eval()
# from optimum.onnxruntime import ORTModelForCausalLM
# model = ORTModelForCausalLM.from_pretrained('sammysun0711/aquilachat-7b-hf', export=True, use_cache=True, trust_remote_code=True)

# from optimum.intel import OVModelForCausalLM
# model = OVModelForCausalLM.from_pretrained('sammysun0711/aquilachat-7b-hf', export=True, use_cache=True, trust_remote_code=True)

question = '北京为什么是中国的首都？'
prompt = (
    '''A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.'''
    f'''###Human: {question}###Assistant:'''
)
with torch.no_grad():
    ret = model.generate(
        **tokenizer(prompt, return_tensors='pt').to('cpu'),
        do_sample=False,
        max_new_tokens=200,
        use_cache=True
    )
    print(tokenizer.decode(ret.tolist()[0]))

北京之所以成为中国的首都，是因为它有着独特的地理位置和历史背景。北京位于华北平原中心，周围是山峦起伏的燕山山脉和太行山脉。它自古以来就是华北地区的政治、文化和经济中心，有着重要的地理位置和战略地位。此外，北京还是中国历史文化的中心，有着丰富的历史遗迹和文化遗产，如故宫、天坛、颐和园等。因此，北京不仅是中国政治、文化和经济中心，也是世界知名的旅游胜地。

AquilaChat-7B开源模型使用《智源Aquila系列模型许可协议》, 原始代码基于Apache Licence 2.0。