Text Generation
Transformers
English
mpt
llm-rs
ggml
text-generation-inference
LLukas22 commited on
Commit
8c63ea9
1 Parent(s): f22be83

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -8
README.md CHANGED
@@ -41,18 +41,13 @@ Via pip: `pip install llm-rs`
41
 
42
  #### Run inference
43
  ```python
44
- from llm_rs import Mpt
45
- from huggingface_hub import hf_hub_download
46
 
47
- #Download the model
48
- hf_hub_download(repo_id="LLukas22/mpt-7b-ggml", filename="mpt-7b-q4_0-ggjt.bin", local_dir=".")
49
-
50
- #Load the model
51
- model = Mpt("mpt-7b-q4_0-ggjt.bin")
52
 
53
  #Generate
54
  print(model.generate("The meaning of life is"))
55
- ```
56
 
57
  ### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
58
 
 
41
 
42
  #### Run inference
43
  ```python
44
+ from llm_rs import AutoModel
 
45
 
46
+ #Load the model, define any model you like from the list above as the `model_file`
47
+ model = AutoModel.from_pretrained("Rustformers/mpt-7b-ggml",model_file="mpt-7b-q4_0-ggjt.bin")
 
 
 
48
 
49
  #Generate
50
  print(model.generate("The meaning of life is"))
 
51
 
52
  ### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
53