Update README.md
Browse files
README.md
CHANGED
|
@@ -39,14 +39,14 @@ They work more or less (sometimes the results are more truthful if the “chat w
|
|
| 39 |
Working well, all other its up to you!
|
| 40 |
<br>
|
| 41 |
|
| 42 |
-
<b>Short hints for using:</b>
|
| 43 |
Set your (Max Tokens)context-lenght 16000t main-model, set your embedder-model (Max Embedding Chunk Length) 1024t,set (Max Context Snippets) usual 14,
|
| 44 |
but in ALLM its cutting all in 1024 character parts, so aprox two times or bit more ~30!
|
| 45 |
-
|
| 46 |
-
-> Ok what that mean
|
| 47 |
|
| 48 |
You can receive 14-snippets a 1024t (14336t) from your document ~10000words and 1600t left for the answer ~1000words (2 pages)
|
| 49 |
-
|
| 50 |
You can play and set for your needs, eg 8-snippets a 2048t, or 28-snippets a 512t ...
|
| 51 |
<ul style="line-height: 1;">
|
| 52 |
<li>8000t (~6000words) ~0.8GB VRAM usage</li>
|
|
@@ -55,8 +55,8 @@ You can play and set for your needs, eg 8-snippets a 2048t, or 28-snippets a 512
|
|
| 55 |
</ul>
|
| 56 |
<br>
|
| 57 |
...
|
| 58 |
-
|
| 59 |
-
<b>How embedding and search works:</b>
|
| 60 |
You have a txt/pdf file maybe 90000words(~300pages). You ask the model lets say "what is described in chapter called XYZ in relation to person ZYX".
|
| 61 |
Now it searches for keywords or similar semantic terms in the document. if it has found them, lets say word and meaning around “XYZ and ZYX” ,
|
| 62 |
now a piece of text 1024token around this word “XYZ/ZYX” is cut out at this point.
|
|
@@ -67,10 +67,11 @@ A question for "summary of the document" is most time not useful, if the documen
|
|
| 67 |
If the documents small like 10-20 Pages, its better you copy the whole text inside the prompt.
|
| 68 |
<br>
|
| 69 |
...
|
| 70 |
-
|
| 71 |
My impression is that at the moment only the first results of the embedder is used (dont ask what happend if you use more than one document at once) so it should be some kind of ranking.
|
| 72 |
<br>
|
| 73 |
...
|
|
|
|
| 74 |
<b>Important -> The Systemprompt:</b>
|
| 75 |
You are a helpful assistant who provides an overview of ... under the aspects of ... .
|
| 76 |
You use attached excerpts from the collection to generate your answers!
|
|
@@ -79,7 +80,7 @@ The context of the entire article should not be given too much weight.
|
|
| 79 |
Answer the user's question!
|
| 80 |
After your answer, briefly explain why you included excerpts (1 to X) in your response and justify briefly if you considered some of them unimportant!
|
| 81 |
(change it for your needs, this example works well when I consult a book about a person and a term related to them)
|
| 82 |
-
|
| 83 |
...
|
| 84 |
<br>
|
| 85 |
<br>
|
|
|
|
| 39 |
Working well, all other its up to you!
|
| 40 |
<br>
|
| 41 |
|
| 42 |
+
<b>Short hints for using:</b><br>
|
| 43 |
Set your (Max Tokens)context-lenght 16000t main-model, set your embedder-model (Max Embedding Chunk Length) 1024t,set (Max Context Snippets) usual 14,
|
| 44 |
but in ALLM its cutting all in 1024 character parts, so aprox two times or bit more ~30!
|
| 45 |
+
<br>
|
| 46 |
+
-> Ok what that mean!<br>
|
| 47 |
|
| 48 |
You can receive 14-snippets a 1024t (14336t) from your document ~10000words and 1600t left for the answer ~1000words (2 pages)
|
| 49 |
+
<br>
|
| 50 |
You can play and set for your needs, eg 8-snippets a 2048t, or 28-snippets a 512t ...
|
| 51 |
<ul style="line-height: 1;">
|
| 52 |
<li>8000t (~6000words) ~0.8GB VRAM usage</li>
|
|
|
|
| 55 |
</ul>
|
| 56 |
<br>
|
| 57 |
...
|
| 58 |
+
<br>
|
| 59 |
+
<b>How embedding and search works:</b><br>
|
| 60 |
You have a txt/pdf file maybe 90000words(~300pages). You ask the model lets say "what is described in chapter called XYZ in relation to person ZYX".
|
| 61 |
Now it searches for keywords or similar semantic terms in the document. if it has found them, lets say word and meaning around “XYZ and ZYX” ,
|
| 62 |
now a piece of text 1024token around this word “XYZ/ZYX” is cut out at this point.
|
|
|
|
| 67 |
If the documents small like 10-20 Pages, its better you copy the whole text inside the prompt.
|
| 68 |
<br>
|
| 69 |
...
|
| 70 |
+
<br>
|
| 71 |
My impression is that at the moment only the first results of the embedder is used (dont ask what happend if you use more than one document at once) so it should be some kind of ranking.
|
| 72 |
<br>
|
| 73 |
...
|
| 74 |
+
<br>
|
| 75 |
<b>Important -> The Systemprompt:</b>
|
| 76 |
You are a helpful assistant who provides an overview of ... under the aspects of ... .
|
| 77 |
You use attached excerpts from the collection to generate your answers!
|
|
|
|
| 80 |
Answer the user's question!
|
| 81 |
After your answer, briefly explain why you included excerpts (1 to X) in your response and justify briefly if you considered some of them unimportant!
|
| 82 |
(change it for your needs, this example works well when I consult a book about a person and a term related to them)
|
| 83 |
+
<br>
|
| 84 |
...
|
| 85 |
<br>
|
| 86 |
<br>
|