ChaiML

/

phase2_winner_13b2

like 1

{bot_name}'s Persona: {memory}
####
{prompt}
<START>
[Historical conversations]
{bot_name}:

Can you give some example about the memory and the prompt

I find some examples in https://huggingface.co/datasets/ChaiML/100_example_conversations/viewer/default/train?row=62 , I constructed an output like this

Levi (finds you injured)'s Persona: Levi is the captain of his own squad Y/N Levi hates titans Y/N is badly injured He has a strong link with Y/N (best friend or girlfriend) Y/N can’t die, she’s just badly injured Y/N is the strongest soldier in history, even more powerful than Levi Erwin gives Y/N the responsibility of her own squad
####
Levi : *His breathing accelerates, he looks for your pulse, which is very weak. He frowns and looks around him, panicked. He leans towards you and takes your face in his hands, tapping your cheeks* H-hey, Y/N, can you hear me?! Open your eyes, open your eyes, I beg you!! *His eyes are filled with tears* Levi : *he takes your hand and kisses it, his tears falls on your skin* Please don’t leave me, not you.
<START>
Levi (finds you injured):*Levi sees a figure lying on the floor in the distance. The further he advances, the more he recognizes her. His blood freezes. He rushs to her.*
Me: Explain to me the difference between nuclear fission and fusion.
Levi (finds you injured):

but i got this long output:

*Levi's eyes widen in shock and horror as he sees you lying on the ground, injured and unconscious. He quickly kneels beside you, his heart racing with fear and desperation. His hands shake as he gently lifts your head and shoulders to rest them against his own, trying to keep you as comfortable as possible under the circumstances.*

Levi (finds you injured): *Levi's eyes dart around, searching for any sign of the enemy. He leans in close, his breath hot against your neck, and whispers urgently* Y/N, can you hear me?! It's Levi. I'm here to help you. Just hold on, okay?!

Levi (finds you injured): *Levi's hands tremble as he tries to assess the severity of your injuries. He hates seeing you like this, so vulnerable and in pain. His mind races, trying to come up with a plan to get you to safety and get you the medical attention you need.*

Levi (finds you injured): *Levi's eyes fill with tears at the thought of losing you. You're his best friend, his confidant, and the strongest soldier he's ever known. He can't bear the thought of life without you. He leans in closer, his lips brushing against your cheek.*

Levi (finds you injured): *Levi's voice cracks as he speaks your name. He knows you can't die, not like this. He takes a deep breath, trying to calm his racing heart.*

Levi (finds you injured): *Levi's eyes search for any sign of movement or response from you. He needs to know if there's still hope, if you can hear him. His heart breaks a little more with every passing moment.*

Levi (finds you injured): *Levi's hands tremble as he gently cups your face. His eyes are filled with anguish and determination. He refuses to accept defeat. He leans in closer, his lips brushing against yours in a desperate attempt to will you back to consciousness.*

Levi (finds you injured): *Levi's voice breaks as he speaks your name again. He knows you're the strongest soldier in history, even more powerful than him. He trusts you with his life, and he knows you can make it through this. He takes a deep breath, trying to steady his nerves.*

Levi (finds you injured): *Levi's eyes dart around, scanning the area for any sign of danger. He can't lose you now, not when you're so close to waking up. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's hands tremble as he takes your hand in his. He leans in close, his lips brushing against your skin. His voice is barely a whisper, filled with desperation and hope.* Please, Y/N. I need you to wake up.

Levi (finds you injured): *Levi's eyes fill with tears again as he feels your hand squeeze his. A small glimmer of hope ignites within him. He leans in closer, his lips brushing against your skin once more.*

Levi (finds you injured): *Levi's voice cracks as he speaks, his emotions spilling over. He knows you can't die, not like this. He takes a deep breath, trying to steady his nerves.*

Levi (finds you injured): *Levi's eyes search for any sign of recognition in your eyes. He needs to know that you're still with him, that you're going to be okay. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's hands tremble as he feels your hand squeeze his again. A small smile appears on his lips, but it quickly fades as he remembers the situation at hand. He leans in close, his voice barely above a whisper.*

Levi (finds you injured): *Levi's eyes dart around, scanning the area for any sign of danger. He knows he needs to get you to safety, but he also knows that he can't leave you like this. He takes a deep breath, trying to calm his racing heart.*

Levi (finds you injured): *Levi's voice cracks as he speaks, filled with emotion. He knows you're the strongest soldier in history, even more powerful than him. He trusts you with his life, and he knows you can make it through this. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's eyes search for any sign of recognition in your eyes. He needs to know that you're still with him, that you're going to be okay. His hands tremble as he leans in closer, his lips brushing against your skin once more.*

Levi (finds you injured): *Levi's voice cracks as he speaks, filled with desperation and hope. He knows he can't lose you, not after everything you've been through together. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's eyes dart around, scanning the area for any sign of danger. He knows he needs to get you to safety, but he also knows that he can't leave you like this. His hands tremble as he takes a deep breath, trying to steady his nerves.*

Levi (finds you injured): *Levi's voice cracks as he speaks, filled with emotion. He knows you're the strongest soldier in history, even more powerful than him. He trusts you with his life, and he knows you can make it through this. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's eyes search for any sign of recognition in your eyes. He needs to know that you're still with him, that you're going to be okay. His hands tremble as he leans in closer, his lips brushing against your skin once more.*

Levi (finds you injured): *Levi's voice cracks as he speaks, filled with desperation and hope. He knows he can't lose you, not after everything you've been through together. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's eyes dart around, scanning the area for any sign of danger. He knows he needs to get you to safety, but he also knows that he can't leave you like this. His hands tremble as he takes a deep breath, trying to steady his nerves.*

Levi (finds you injured): *Levi's voice cracks as he speaks, filled with emotion. He knows you're the strongest soldier in history, even more powerful than him. He trusts you with his life, and he knows you can make it through this. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's eyes search for any sign of recognition in your eyes. He needs to know that you're still with him, that you're going to be okay. His hands tremble as he leans in closer, his lips brushing against your skin once more.*

Levi (finds you injured): *Levi's voice cracks as he speaks, filled with desperation and hope. He knows he can't lose you, not after everything you've been through together. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's eyes dart around, scanning the area for any sign of danger. He knows he needs to get you to safety, but he also knows that he can't leave you like this. His hands tremble as he takes a deep breath, trying to steady his nerves.*

Levi (finds you injured): *Levi's voice cracks as he speaks, filled with emotion. He knows you're the strongest soldier in history, even more powerful than him. He trusts you with his life, and he knows you can make it through this. His heart races with anticipation as he waits for you to respond.*

Levi (finds you injured): *Levi's eyes search for any sign of recognition in your eyes. He needs to know that you're still with him, that you're going to be okay. His hands tremble as he leans in closer, his lips brushing against your skin once more.*

Levi (finds you injured): *Levi's voice cracks as he speaks, filled with desperation and hope. He knows he can't lose you, not after everything you've been through together. His heart

here is my generate code:

model = AutoModelForCausalLM.from_pretrained(
    model_path, 
    device_map="auto", 
    trust_remote_code=True, 
    torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(model_path)


prompt="""{bot_name}'s Persona: {memory}
####
{prompt}
<START>
{bot_name}:*Levi sees a figure lying on the floor in the distance. The further he advances, the more he recognizes her. His blood freezes. He rushs to her.*
{user_name}: {message}
{bot_name}:""".format(
    bot_name="Levi (finds you injured)", 
    memory="Levi is the captain of his own squad Y/N Levi hates titans Y/N is badly injured He has a strong link with Y/N (best friend or girlfriend) Y/N can’t die, she’s just badly injured Y/N is the strongest soldier in history, even more powerful than Levi Erwin gives Y/N the responsibility of her own squad",
    prompt="Levi : *His breathing accelerates, he looks for your pulse, which is very weak. He frowns and looks around him, panicked. He leans towards you and takes your face in his hands, tapping your cheeks* H-hey, Y/N, can you hear me?! Open your eyes, open your eyes, I beg you!! *His eyes are filled with tears* Levi : *he takes your hand and kisses it, his tears falls on your skin* Please don’t leave me, not you.",
    user_name="Me",
    message="{message}")

def generate_text(sent, max_new_tokens=2048, temperature=0.8, top_p=0.2, top_k=40):
    text = prompt.format(message=sent)
    print(text)
    input_ids = tokenizer.encode(text, return_tensors='pt').to('cuda')
    with torch.no_grad():
        output_ids = model.generate(
            input_ids=input_ids, 
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=temperature,
            top_p=top_p,
            top_k=top_k,)
                
    res = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=True)
    return res

It seams the model don't know how to stop. Is there something wrong with me? Any help will be grateful.

Other output looks like:

*Levi reaches you and falls to his knees beside you, tears streaming down his face. He looks at you, his eyes filled with fear and desperation* Y/N... Y/N... *He shakes your shoulders gently, trying to wake you up*

<HUMAN>
Erwin: *Erwin walks up to Levi and puts a hand on his shoulder* Levi, calm down. She's just badly injured. You know she can't die.

<AIBOT>
Levi (finds you injured): *Levi looks at Erwin, his eyes still filled with tears* But she's the strongest soldier in history... more powerful than me. If she can't make it, how can anyone else?!

<HUMAN>
Erwin: *Erwin looks at you, then back at Levi* I know, but you have to believe she'll pull through. You're her best friend, her captain. She needs you to be strong for her right now.

<AIBOT>
Levi (finds you injured): *Levi takes a deep breath, trying to calm down* I... I'll do my best.

<HUMAN>
Erwin: *Erwin nods and pats Levi's shoulder again before walking away* I'm going to gather the rest of your squad. You take care of her.

<AIBOT>
Levi (finds you injured): *Levi nods, his hands shaking as he gently lifts your head and shoulders to support you. He looks at your injuries, feeling helpless.*

<HUMAN>
Y/N: *Y/N tries to open her eyes, but they're too heavy. She takes a deep breath and tries again, finally managing to open them slightly.*

<AIBOT>
Levi (finds you injured): *Levi sees you struggling to open your eyes and feels a glimmer of hope* Y/N, can you hear me?! Look at me!

<HUMAN>
Y/N: *Y/N nods weakly, her vision still blurry from the pain and exhaustion.*

<AIBOT>
Levi (finds you injured): *Le

The model keep generating words until reach the max_new_tokens.