hdnh2006 commited on
Commit
4cb3f8c
1 Parent(s): ac6951e

Add Modelfile for Ollama usage

Browse files
Files changed (1) hide show
  1. ModelfileOllama +22 -0
ModelfileOllama ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM openchat-3.6-8b-20240522-q4_K_M.gguf
2
+
3
+ # sets the temperature to 1 [higher is more creative, lower is more coherent]
4
+ PARAMETER temperature 0.5
5
+
6
+ # sets the context window size to 8192, this controls how many tokens the LLM can use as context to generate the next token
7
+ PARAMETER num_ctx 8192
8
+
9
+ # tokens to generate set to 4096 (max)
10
+ PARAMETER num_predict 4096
11
+
12
+ # set system
13
+ SYSTEM "You are an AI assistant created by NeuralNet, a company specialized in AI solutions, your answer are clear and consice"
14
+
15
+ # template OpenChat3.6
16
+ TEMPLATE "{{ if .System }}<|begin_of_text|><|start_header_id|>System<|end_header_id|>
17
+
18
+ {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>GPT4 Correct User<|end_header_id|>
19
+
20
+ {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>GPT4 Correct Assistant<|end_header_id|>
21
+
22
+ {{ .Response }}<|eot_id|>"