Spaces:
Runtime error
Runtime error
Can you share the full deploy code
#1
by
tomriddle
- opened
Can you share the full deployment code which you have deployed on other server and using end point to confirm that this space is miqu
- Download ollama from github
- Create Modelfile card
# Modelfile for a Q&A model with creative responses
FROM ./miqu.gguf
# Setting the temperature to 1 for more creative responses
PARAMETER temperature 1
PARAMETER top_p 0.95
# Template for processing inputs in the format of <s> [INST] QUERY_1 [/INST] ANSWER_1</s> [INST] QUERY_2 [/INST] ANSWER_2</s>...
TEMPLATE """{{- if .First -}}<s>{{- end -}}[INST] {{ .Prompt }} [/INST] {{ .Response }}</s> """
Download Miqu.gguf with weights of that model (You can find link here )
Start ollama serving (ollama serve)
Replace LLM service url to your own server IP
Enjoy!