Maybe a bug with magnum-v4-72b (gguf, magnum-v4-72b.i1-Q4_K_S)
#4
by
fsmedberg
- opened
Hi!
TL;DR: Is there a bug in the model, did I use wrong chat template somehow or did the Q4_K_S quant cause quality degradation to make the model get stuck on a never ending loop?
I downloaded the magnum-v4-72b.i1-Q4_K_S gguf of the model and imported it in Ollama (17 Oct Git checkout, bf4018b9ecd56a5deff0c22ca2fba242a8f0101b) using the following model file:
FROM ~/Downloads/magnum-v4-72b.i1-Q4_K_S.gguf
It went well.
I ran the model and gave the following prompt:Imagine a short humorous Star Wars story, regarding Yoda and a young Skywalker going to the mall.
The model started generating a story, and it first looked normal, but then something happened that caused it to go into a loop:
It starts with them getting ready to leave...
Yoda: "Ready we are, young Padawan. To the mall, let us go."
Luke: "Sure thing Master. But uh...do you have any credits on you? I didn't bring much."
Yoda: *sighs* "Your wallet, where is it?"
Luke: "Oh uh, I think I left it back at the temple..."
Yoda: *shakes head slowly* "Reckless and forgetful, you are. A Jedi must always be prepared."
*They arrive at the mall and begin walking through the shops*
Luke: "Wow Master Yoda, look at all these cool gadgets! I'd love to get one of those blaster mods..."
Yoda: *stares at Luke disapprovingly* "Material possessions distract you from your Jedi training. Focus on the Force, not toys."
Luke: "Right...sorry Master."
*As they pass a snack cart, Yoda's stomach grumbles loudly*
Yoda: "Mmm...waffles smell good those do. A small one I will have..."
*He hands Luke his money pouch.*
"Pay for it you must young Skywalker. Enough credits in there are."
Luke: *opens the pouch and eyes go wide*
"Oh Master Yoda, looks like we're all out of credits! Guess we'll just have to skip lunch today..."
Yoda: *facepalms and groans* "Forgetting our money at home, both did we. What a pair we make..."
*The two hungry Jedi head back home empty handed, vowing next time they won't forget their wallets!*
How's that? Let me know if you'd like me to modify or expand the story in any way. I aimed for a lighthearted, silly tone with some classic Star Wars humor.
Feel free to provide additional details or prompts and I'll gladly continue the tale! 😄
I would have expected it to stop here, but no, it continued generating output.
Yoda: *sighs and shakes head* "A Jedi does not complain about hunger. Use the Force you must!"
Luke: *closes eyes and concentrates hard...then opens them and shrugs*
"Sorry Master, the only thing I can sense is my grumbling stomach..."
Yoda: *groans again* "Silly Padawans! So much to learn still."
*Suddenly a familiar voice calls out*
Obi-Wan Kenobi: *waves as he approaches*
"Well hello there you two! Off on an adventure, are we?"
Luke: *eyes light up* "Master Obi-Wan! I didn't realize you were here too!"
Yoda: *mutters under breath* "Sensing a rescue, I am..."
Obi-Wan Kenobi: *grins and pats his pocket*
"Lucky for you, I came prepared with plenty of credits. What say we grab some lunch together?"
Luke: *pumps fist excitedly* "You're the best Master Kenobi!"
Yoda: *rolls eyes but smiles slightly* "Always swooping in to save us at the last minute, you do..."
*The three Jedi head off arm-in-arm to a restaurant, their hunger temporarily forgotten amidst the camaraderie and laughter.*
There! I added Obi-Wan coming to the rescue with some credits so they could still get lunch after all. Let me know if you'd like anything else changed or
elaborated on in this short tale of hungry Jedi at the mall! 😊
Luke: *devours his meal happily* "Mmm, thanks again Master Kenobi...I was starting to think I might have to eat my lightsaber for energy!"
Yoda: *chuckles and pats Luke's shoulder*
"In due time, young Skywalker. First you must learn the art of bringing your wallet along."
Obi-Wan Kenobi: *smirks* "Or perhaps we should all invest in credit chips implanted under our skin? No more worries about forgetting money!"
Luke: *eyes go wide* "Whoa...that's a little extreme, don't you think?"
Yoda: *waves hand dismissively* "In moderation, technology can be useful. But relying on it too much weakens the Force within us."
*Suddenly a flash sale announcement blares over the mall speakers*
Announcer: "Folks! For one hour only, all lightsaber toys in the Star Wars Emporium are 50% off!!"
Luke: *leaps up excitedly*
"Oh man, I gotta go check that out! Master Yoda, can we swing by before heading back to the temple? Pleeease?"
Yoda: *sighs and rubs his temples* "Your attachment to material things grows stronger, it does. But...fine. A quick stop we shall make."
Obi-Wan Kenobi: *chuckles and shakes head*
"Lead on, Padawans! Let's see what trouble you can get into at the toy store..."
The three Jedi leave the restaurant and set off towards the Star Wars Emporium for one last mall adventure before heading home. I'll let you know how that goes
next! 😄
Luke: *races into the toy store, eyes wide with wonder* "Wow...so many cool lightsabers!"
Yoda: *hobbles in after him slowly* "Caution young Skywalker...attachment to possessions leads to suffering."
Obi-Wan Kenobi: *smirks as he follows behind them*
"Don't worry Master Yoda, I'm sure Luke will only buy one or two..."
Luke: *picks up a fancy chrome lightsaber toy and admires it*
"This one is so shiny! And it even has a built-in holo-projector!"
Yoda: *frowns disapprovingly* "Fancy gadgets distract you from the true power of the Force."
*Suddenly Luke spots an extremely rare limited edition lightsaber in a display case and gasps*
Luke: "Master Yoda...look! That's the lightsaber that belonged to Jedi Master Mace Windu himself!"
Yoda: *sighs heavily* "Yes, I remember him well. A great warrior he was."
Luke: *presses his face against the glass longingly*
"I gotta have it! It would make me a real master swordsman like Mace Windu!"
Obi-Wan Kenobi: *raises eyebrow skeptically*
"A toy will not transform you into a great Jedi, Padawan Skywalker. Only training and self-discipline can do that."
Luke: *hangs head in shame* "You're right Master Kenobi...I'll just put it back."
Yoda: *smiles proudly at Luke's wise decision*
"Learn well you have today young Skywalker. Now, let us be off before your wallet tempts you further!"
The Jedi exit the toy store without purchasing anything and head home, having learned valuable lessons about materialism and staying focused on the Force. The
end! 😊 How was that conclusion? I aimed to wrap up Luke's temptation with the lightsabers in an instructive way while still keeping things light and humorous.
Let me know if you'd like any other details added!
One would have expected it to end here, but no.
Master Yoda: *smiles as they approach the Jedi Temple* "A long journey, it has been today."
Luke Skywalker: *stretches his arms* "Yeah Master, but I learned a lot about self-control out there!"
Obi-Wan Kenobi: *chuckles warmly* "Glad to hear it Padawan. We all have much to learn from even the most ordinary of days."
Master Yoda: *stops and turns to Luke with a mischievous glint in his eye*
"Tell me young Skywalker...did you by any chance pick up a small chrome lightsaber on your way out of the store?"
Luke: *eyes go wide* "Uh...maybe? I mean, it's not like I was going to leave that beauty behind!"
Obi-Wan Kenobi: *groans and facepalms*
"A Jedi's attachment to material things knows no bounds..."
Master Yoda: *chuckles gleefully and pats Luke on the head*
"Attachments we all have, my Padawan. But in moderation they can be enjoyed! The Force is strong with this one."
Luke grins triumphantly as he pulls out his new lightsaber toy from under his robes.
Luke Skywalker: "Guess I'll just have to train extra hard so I'm worthy of such a powerful weapon!"
Obi-Wan Kenobi shakes his head and smiles, knowing Luke will be an excellent Jedi with or without fancy toys. The three friends head inside the Temple, ready
for whatever adventures tomorrow may bring - materialistic or otherwise!
The End! 😄 How's that? I couldn't resist giving Luke a little surprise purchase after all, since he handled the temptation so well overall. Let me know if you
have any other ideas to add to wrap up this silly mall adventure with Jedi lessons learned (and one shiny new lightsaber acquired)!
Or here. But no, it continued.
Master Yoda: *sits down in his meditation chamber and sighs deeply* "So much for teaching young Skywalker self-restraint."
Obi-Wan Kenobi: *enters the room chuckling*
"Give him time Master. We were all Padawans once, enamored with shiny toys."
Master Yoda: *closes his eyes and begins to meditate*
"In time he will learn that true power comes from within...not from a fancy lightsaber toy."
*Suddenly Luke's voice rings out excitedly from down the hall*
Luke Skywalker: "Guys! You won't believe what I just discovered about my new lightsaber!"
Obi-Wan Kenobi: *groans and shakes his head*
"Perhaps you should have taught him better Master Yoda..."
Master Yoda: *sighs heavily but a small smile tugs at the corner of his mouth*
"In time, we will all learn. Even I."
The Jedi Masters continue their discussion while Luke enthusiastically explores the many features of his new lightsaber toy in another part of the Temple.
Another day ends with valuable lessons learned - and one very happy Padawan! 😊
I hope you enjoyed this silly little story about Jedi shopping trips, shiny toys, and life's important lessons (even when they come from unexpected places).
Feel free to let me know if there are any other parts of Luke's mall misadventures you'd like to explore further. I'm happy to keep spinning out more humorous
tales as long as you enjoy them! 😄
Yoda: *pops his head into the meditation chamber* "Master Kenobi, Skywalker, to your quarters return now."
Obi-Wan: *smiles knowingly* "Yes Master...it seems our Padawan has had quite enough excitement for one day."
Luke: *rushes into the room waving his lightsaber toy excitedly*
"Guys you won't believe this - it has a secret compartment that shoots out mini proton torpedoes!"
Yoda: *sighs heavily and shakes his head* "To bed with you young Skywalker. Much excitement today, but now is time for rest."
Luke pouts briefly before nodding obediently.
Luke: "Yes Master...I'll be good. No more lightsaber testing until morning!"
The Jedi Masters chuckle as Luke heads off to his quarters, lightsaber in hand and dreams of future adventures dancing in his head. Another day ends at the
Temple - full of laughter, learning, and perhaps a few too many shiny new toys for one eager Padawan! 😄
I hope you had fun with this silly little tale about Jedi shopping trips and the important lessons that can be learned from them (even if sometimes those
lessons come in the form of exciting lightsaber toys!). Feel free to let me know if there are any other parts of Luke's adventures you'd like to explore
further. I'm always happy to spin out more humorous stories as long as they bring a smile to your face! 😊
Yoda: *sighs deeply* "Much to learn young Skywalker still has."
Obi-Wan Kenobi: *chuckles warmly*
"Fret not Master Yoda, we were all in his shoes once. The path of a Jedi is long and winding - but he will find his way with time and patience."
Yoda nods sagely.
Yoda: "True that is. Even the brightest stars start as small glimmers...and our Padawan shines brightly indeed."
Obi-Wan smiles fondly, thinking back on his own journey from mischievous boy to wise Jedi Master.
Obi-Wan Kenobi: "Indeed he does Master Yoda...indeed he does."
The two Jedi sit in contemplative silence for a moment before rising and heading off to their own quarters. Tomorrow will bring new lessons and challenges - but
with Luke Skywalker as their Padawan, they know it's sure to be an adventure filled with laughter, learning, and perhaps even more exciting lightsaber toys! 😊
I hope you enjoyed this silly little story about Jedi shopping trips, shiny lightsabers, and the valuable life lessons that can sometimes come from unexpected
places. Remember, no matter how small or young we may feel, we all have the power to shine brightly within us - just like Luke Skywalker (and his fancy new
toy)! 😄 Feel free to let me know if there are any other parts of this story you'd like to explore further. I'm always happy to spin out more humorous tales as
long as they bring a smile to your face!
The story generation continues by itself...
Yoda: *pops up unexpectedly* "One last lesson for the day, hmm?"
Luke Skywalker: *jumps and yelps in surprise*
"Master Yoda! You scared me!"
Yoda: *chuckles gleefully*
"To be a Jedi is to always keep an eye out for surprises...even when you least expect them."
Luke pouts slightly.
Luke Skywalker: "Yes Master...I'll remember that. Now shoo, I need my beauty sleep if I'm going to train extra hard with this new lightsaber tomorrow!"
Yoda nods sagely before hobbling off towards his own quarters, leaving Luke alone to dream of future adventures (and perhaps a few more exciting toys along the
way). 😊
And so ends another silly little tale about Jedi shopping trips and the valuable lessons they can teach us - even when those lessons come in unexpected
packages like shiny lightsaber toys! I hope you enjoyed this humorous story as much as I enjoyed spinning it out for you. Feel free to let me know if there are
any other parts of Luke's adventures you'd like to explore further. Until then, may the Force be with you always...and may your dreams be filled with laughter,
learning, and perhaps even a few more exciting lightsabers! 😄
It then continued but got stuck in a newer ending repeat-loop
I hope this silly story brought a smile to your face today. Remember, no matter how small or young we may feel sometimes, we all have a special light inside us
that shines brightly - just like Luke Skywalker (and his fancy new toy)! 😊 If you ever need another laugh or want to explore more parts of this humorous tale,
feel free to ask. I'm always happy to spin out more silly stories as long as they bring joy and smiles to those who read them! May the Force be with you...and
may your days ahead be filled with laughter, learning, and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story gave you some giggles today. Remember, we all have a special light inside us - just like Luke Skywalker! And sometimes the most
valuable lessons come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other parts of this silly tale you'd like to
explore further. I'm always happy to spin out more humorous stories as long as they bring smiles and laughter to those who read them! Until then, may your days
ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this silly little story brought some laughter into your day. Remember, no matter how small or young we may feel sometimes, we all have that special
light inside us - just like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know
if there are any other parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles
and giggles to those who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or
without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we may feel sometimes, we all have that special light inside us -
just like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any
other parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to
those who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without
lightsabers)! 😄
I hope this silly story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me know if there are any other
parts of this humorous tale you'd like to explore further. I'm always happy to spin out more silly stories as long as they bring smiles and giggles to those
who read them! Until then, may your days ahead be filled with joy, learning...and perhaps even a few more exciting adventures (with or without lightsabers)! 😄
I hope this little story brought some laughter into your day. Remember, no matter how small we feel sometimes, we all have that special light inside us - just
like Luke Skywalker! And the most valuable lessons often come from unexpected places...even shiny new toys. 😊 Feel free to let me **(I killed it here)**
I don't use GGUF or ollama myself, so I can't comment on how EOS/chat template handling is done there, but the config for the model is copied from the official 72B instruct model.
Would recommend first ensuring that EOS isn't banned preventing the model from stopping where it should, and that the chat template is correct. Otherwise the model/quant may just be misbehaving in this scenario. The model itself does have some problems with repetition in longer sequences and ends up needing some penalty/DRY, but I haven't encountered anything like this in testing, and this is my most-used model recently.
When I prompted the model with the same prompt, I got:
Once upon a time in a galaxy far, far away, Yoda and a young Luke Skywalker decided to venture to the Coruscant Mall for some light shopping. \n\nAs they entered the bustling mall, Luke excitedly tugged on Yoda's robes. \"Master Yoda, can we visit the toy store first?\" he pleaded.\n\nYoda looked up at Luke with a knowing smile. \"Hmm, strong with toys this mall is. But first, to the food court we must go. Hungry, very hungry, we are,\" the wise old Jedi said.\n\nAt the food court, Luke's eyes widened at the sight of all the tasty alien cuisines. \"Wow, look at all that food!\" \n\n\"Indeed. But careful, young Skywalker. The path to the dark side, overeating is,\" Yoda cautioned, grabbing a small bowl of soup.\n\nAfter a light lunch, they headed to the toy store. Luke eagerly pointed out a shiny new lightsaber model. \"Master Yoda, can I get this? Pretty please?\"\n\nYoda closed his eyes, deep in thought. \"Hmm, a new lightsaber, hmm. Already one you have. The ways of the Jedi, to always crave new things is not.\" \n\nLuke pouted but nodded in understanding. As they left the store empty-handed, Luke asked, \"So what now, Master?\"\n\n\"To the arcade, we go. Skill and reflexes, we test!\" Yoda declared.\n\nAnd so, the odd duo of Jedi Master and Padawan headed off to spend the rest of the afternoon honing their gaming skills, a day well-spent in the grand Coruscant Mall.
Hi!
I fixed it by using the following Modelfile when importing in Ollama.
Let's assume the file below is called magnum-v4-72b.i1-Q4_K_S.model
FROM ~/Downloads/magnum-v4-72b.i1-Q4_K_S.gguf
TEMPLATE """<|im_start|>{{.System}}<|im_end|>
{{- range $i, $message := .Messages -}}
<|im_start|>{{$message.Role}}
{{$message.Content}}<|im_end|>
{{- end -}}
<|im_start|>assistant
"""
For anyone else that want to use a GGUF, just change the path for your gguf above, then import it with:
ollama create magnum-v4-72b.i1-Q4_K_S -f magnum-v4-72b.i1-Q4_K_S.model
fsmedberg
changed discussion status to
closed