legraphista

/

Higgs-Llama-3-70B-IMat-GGUF

like 7

hey @gopi87 ,

I've just tested it in the latest llama.cpp and it appears to work:

$ llama.cpp/main -m Higgs-Llama-3-70B.Q2_K.gguf -ngl 60 --special --prompt "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nwhat is 1337 ^ 2?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"


<|begin_of_text|><|begin_of_text|><|start_header_id|>user<|end_header_id|>

what is 1337 ^ 2?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

1337^2 is equal to 1,785,769. This is because when you multiply any number by itself (in this case 1337), you are squaring that number. So 1337 x 1337 results in the calculation of 1,785,769. This value represents the area of a square with sides of length 1337.<|eot_id|> [end of text]

Can you confirm that you're running on the latest koboldcpp?

As a sidenote, I did observe that the model at low quants refused to generate text if the header of assistant (<|start_header_id|>assistant<|end_header_id|>\n\n) was not present in the prompt. Maybe you can try adding that to your prompt

hi sorry for this its working well in kolboldcpp