Transformers
GGUF
llama
TheBloke commited on
Commit
5b4b3b4
1 Parent(s): 7868c3a

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -169,7 +169,7 @@ CT_METAL=1 pip install ctransformers>=0.2.24 --no-binary ctransformers
169
  from ctransformers import AutoModelForCausalLM
170
 
171
  # Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
172
- llm = AutoModelForCausalLM.from_pretrained("TheBloke/Yarn-Llama-2-7B-128K-GGML", model_file="yarn-llama-2-7b-128k.q4_K_M.gguf", model_type="llama", gpu_layers=50)
173
 
174
  print(llm("AI is going to"))
175
  ```
 
169
  from ctransformers import AutoModelForCausalLM
170
 
171
  # Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
172
+ llm = AutoModelForCausalLM.from_pretrained("TheBloke/Yarn-Llama-2-7B-128K-GGUF", model_file="yarn-llama-2-7b-128k.q4_K_M.gguf", model_type="llama", gpu_layers=50)
173
 
174
  print(llm("AI is going to"))
175
  ```