Model broken?

by wcde - opened Apr 25

Discussion

wcde

Apr 25

I tried several quants, all of them produce random set of characters. EXL2 works without problems.

bartowski

Owner Apr 26

@wcde Have you double checked your prompt format and more importantly the rope scale?

this one needs the rope scale set to 4, i'm not sure why llama.cpp seems to have ignored that during conversion.. it produces pure gibberish if the rope scale isn't set properly

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment