YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
🤭 Please refer to https://github.com/svjack/Genshin-Impact-Character-Chat to get more info
Run vLLM Server
python -m vllm.entrypoints.openai.api_server --model svjack/DPO_Genshin_Impact_Mistral_Plot_Engine_Step_Json_Short_AWQ --dtype auto \
--api-key token-abc123 --quantization awq --max-model-len 6000 --gpu-memory-utilization 0.9
Run Client
- Step 1: Generate a story Backgroud In Genshin Impact
from openai import OpenAI
from IPython.display import clear_output
# Set OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "token-abc123"
openai_api_base = "http://localhost:8000/v1"
# Create an OpenAI client to interact with the API server
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
stream = client.chat.completions.create(
model="svjack/DPO_Genshin_Impact_Mistral_Plot_Engine_Step_Json_Short_AWQ", # Model name to use
messages=[
{
"role" :"system",
"content": ""
},
{
"role": "user",
"content": '''
故事标题:归乡
故事背景:在须弥城门口,派蒙与纳西妲偶遇并帮助一只昏迷的元素生命找寻家园。过程中揭示了这只生物并非普通的蕈兽,而是元素生物,并且它们曾受到过‘末日’的影响,家园被侵蚀。纳西妲回忆起晶体里的力量可能与一个预言有关,为了拯救它们的家园,她必须解决‘禁忌知识’问题,但这个过程对她自身也会产生干扰。
参与角色:派蒙、纳西妲、浮游水蕈兽、旅行者
'''
},
], # Chat history
temperature=0.01, # Temperature for text generation
stream=True, # Stream response
)
# Read and return generated text from response stream
partial_message = ""
for chunk in stream:
clear_output(wait = True)
partial_message += (chunk.choices[0].delta.content or "")
print(partial_message)
Output
{'参与者1': '派蒙', '参与者2': '纳西妲', '当前故事背景': '在须弥城门口,派蒙发现了一只昏迷的浮游水蕈兽,并询问它的情况。纳西妲解释这只生物是元素生物,并且它们曾经有家园,但被‘末日’影响。纳西妲决定帮助它们,并提到她需要解决‘禁忌知识’以拯救它们。'}
def openai_predict(messages, temperature = 0.01):
stream = client.chat.completions.create(
model="svjack/DPO_Genshin_Impact_Mistral_Plot_Engine_Step_Json_Short_AWQ", # Model name to use
messages=messages, # Chat history
temperature=temperature, # Temperature for text generation
stream=True, # Stream response
)
# Read and return generated text from response stream
partial_message = ""
for chunk in stream:
clear_output(wait = True)
partial_message += (chunk.choices[0].delta.content or "")
print(partial_message)
return partial_message
from rapidfuzz import fuzz
from IPython.display import clear_output
def run_step_infer_times(x, times = 5, temperature = 0.01,
repetition_penalty = 1.0,
sim_val = 70
):
req = []
for _ in range(times):
clear_output(wait = True)
out = openai_predict([
{
"role": "system",
"content": ""
},
{
"role": "user",
"content": x
},
],
#repetition_penalty = repetition_penalty,
temperature = temperature,
#max_new_tokens = 2070,
#max_input_length = 6000,
)
if req:
val = max(map(lambda x: fuzz.ratio(x, out), req))
#print(val)
#print(req)
if val < sim_val:
req.append(out.strip())
x = x.strip() + "\n" + out.strip()
else:
req.append(out.strip())
x = x.strip() + "\n" + out.strip()
return req
out_l = run_step_infer_times(
'''
故事标题:归乡
故事背景:在须弥城门口,派蒙与纳西妲偶遇并帮助一只昏迷的元素生命找寻家园。过程中揭示了这只生物并非普通的蕈兽,而是元素生物,并且它们曾受到过‘末日’的影响,家园被侵蚀。纳西妲回忆起晶体里的力量可能与一个预言有关,为了拯救它们的家园,她必须解决‘禁忌知识’问题,但这个过程对她自身也会产生干扰。
参与角色:派蒙、纳西妲、浮游水蕈兽、旅行者
''',
temperature=0.1,
repetition_penalty = 1.0,
times = 5
)
clear_output(wait = True)
print("\n".join(out_l))
Output
{'参与者1': '派蒙', '参与者2': '纳西妲', '当前故事背景': '在须弥城门口,派蒙发现了一只昏迷的浮游水蕈兽,并询问它的情况。纳西妲解释这只生物是元素生物,并且它们曾经有家园,但被‘末日’影响。纳西妲提到晶体中的力量可能与一个预言有关,她需要解决‘禁忌知识’以帮助它们。'}
{'参与者1': '派蒙', '参与者2': '浮游水蕈兽', '当前故事背景': '派蒙询问浮游水蕈兽是否有家,发现它并不记得,这让纳西妲推测它的家园可能被侵蚀。'}
{'参与者1': '纳西妲', '参与者2': '旅行者', '当前故事背景': '纳西妲解释了浮游水蕈兽的元素生命以及它们的家园,并提到‘末日’对它们的影响。她提到晶体中的力量可能与一个预言有关联,并请求旅行者帮助她解决‘禁忌知识’问题。'}
{'参与者1': '纳西妲', '参与者2': '浮游水蕈兽', '当前故事背景': '纳西妲试图通过晶体的力量寻找线索,但发现它并不能提供直接的答案,这让她感到困扰。'}
{'参与者1': '派蒙', '参与者2': '纳西妲', '当前故事背景': '派蒙对纳西妲的‘禁忌知识’感到好奇,并询问其含义,纳西妲解释这是一个可能导致她的记忆混乱的力量。'}
- Downloads last month
- 8
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.