Not able to run LongWriter-llama3.1-8b-Q8_0-GGUF on Ollama
I built the ollama model for the LongWriter-llama3.1-8b-Q8_0-GGUF with the modelfile as below, and it doesn't output answers when run in ollama. Please help me fix this or point out what I am doing wrong. I have very less knowledge of coding, and I mostly will be using these models for academic work. Thanks in advance!
ModelFile contents -
FROM C:/Users/Prajw/.ollama/models/modelsfiles/longwriter/longwriter-llama3.1-8b-q8_0.gguf
PARAMETER temperature 0.5
PARAMETER top_p 0.8
PARAMETER top_k 50
PARAMETER num_ctx 32768
PARAMETER repeat_penalty 1.15
PARAMETER stop USER:
PARAMETER stop ASSISTANT:
TEMPLATE """{{ if .System }}
<>
{{ .System }}
<>
{{ end }}
[INST]{{ index .Messages 0 "Content" }}[/INST]{{ index .Messages 0 "Response" }}
{{ range $i, $msg := .Messages }}
{{- if ne $i 0 -}}
[INST]{{ $msg.Content }}[/INST]{{ $msg.Response }}
{{- end -}}
{{ end }}"""