jartine commited on
Commit
d8802fc
1 Parent(s): 64ea811

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -55,10 +55,9 @@ chmod +x Meta-Llama-3.1-8B-Instruct.Q6_K.llamafile
55
  You then need to fill out the prompt / history template (see below).
56
 
57
  This model has a max context window size of 128k tokens. By default, a
58
- context window size of 512 tokens is used. You may increase this to the
59
- maximum by passing the `-c 0` flag, which uses considerable memory. A
60
- context window of `-c 32768` ought to be big enough for most use cases
61
- and it incurs significantly less memory overhead.
62
 
63
  On GPUs with sufficient RAM, the `-ngl 999` flag may be passed to use
64
  the system's NVIDIA or AMD GPU(s). On Windows, only the graphics card
 
55
  You then need to fill out the prompt / history template (see below).
56
 
57
  This model has a max context window size of 128k tokens. By default, a
58
+ context window size of 512 tokens is used. You can use a larger context
59
+ window by passing the `-c 8192` flag. The software currently has
60
+ limitations that may prevent scaling to the full 128k size.
 
61
 
62
  On GPUs with sufficient RAM, the `-ngl 999` flag may be passed to use
63
  the system's NVIDIA or AMD GPU(s). On Windows, only the graphics card