QuantFactory

/

Hebrew-Mistral-7B-GGUF

like 1

Follow

Quant Factory 424

Hi , i can see that there is no template for this model, and when i use it locally (Downloading and run it) i am getting random responses.
I created Modelfile and copy the core mistral template into it , but still i am getting random responses.

Am i missing anything ?
Should i use any other template ?

my model file :

FROM hf.co/QuantFactory/Hebrew-Mistral-7B-GGUF:Q4_K_M

TEMPLATE """{{- if .Messages }}
{{- range $index, $_ := .Messages }}
{{- if eq .Role "user" }}
{{- if and (eq (len (slice $.Messages $index)) 1) $.Tools }}[AVAILABLE_TOOLS] {{ $.Tools }}[/AVAILABLE_TOOLS]
{{- end }}[INST] {{ if and $.System (eq (len (slice $.Messages $index)) 1) }}{{ $.System }}

{{ end }}{{ .Content }}[/INST]
{{- else if eq .Role "assistant" }}
{{- if .Content }} {{ .Content }}
{{- else if .ToolCalls }}[TOOL_CALLS] [
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}]
{{- end }}
{{- else if eq .Role "tool" }}[TOOL_RESULTS] {"content": {{ .Content }}} [/TOOL_RESULTS]
{{- end }}
{{- end }}
{{- else }}[INST] {{ if .System }}{{ .System }}

{{ end }}{{ .Prompt }}[/INST]
{{- end }} {{ .Response }}
{{- if .Response }}
{{- end }}"""

PARAMETER stop ""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER temperature 0