Text Generation
Transformers
English
mpt
llm-rs
ggml
text-generation-inference
LLukas22 commited on
Commit
4e562e7
1 Parent(s): 4aaf40f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -3
README.md CHANGED
@@ -13,8 +13,7 @@ tags:
13
 
14
 
15
  ## CAUTION: MPT Development is still ongoing and not finished!
16
- - GGML implementation see here: [Replit + MPT](https://github.com/ggerganov/ggml/pull/145)
17
- - Rustformers implementation see here: [Implement MPT Model](https://github.com/rustformers/llm/pull/218)
18
 
19
  If these implementations are complete i will add instructions on how to run the models and update them if necesary!
20
 
@@ -32,4 +31,28 @@ If these implementations are complete i will add instructions on how to run the
32
  | [mpt-7b-instruct-q4_0-ggjt.bin](https://huggingface.co/LLukas22/mpt-7b-ggml/blob/main/mpt-7b-instruct-q4_0-ggjt.bin) | [mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) | int4 | GGJT |
33
  | [mpt-7b-storywriter-f16.bin](https://huggingface.co/LLukas22/mpt-7b-ggml/blob/main/mpt-7b-f16.bin) | [mpt-7b-storywriter](https://huggingface.co/mosaicml/mpt-7b-storywriter) | fp16 | GGML |
34
  | [mpt-7b-storywriter-q4_0-ggjt.bin](https://huggingface.co/LLukas22/mpt-7b-ggml/blob/main/mpt-7b-storywriter-q4_0-ggjt.bin) | [mpt-7b-storywriter](https://huggingface.co/mosaicml/mpt-7b-storywriter) | int4 | GGJT |
35
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
 
15
  ## CAUTION: MPT Development is still ongoing and not finished!
16
+ - Rust & Python: Rustformers implementation see here: [Implement MPT Model](https://github.com/rustformers/llm/pull/218)
 
17
 
18
  If these implementations are complete i will add instructions on how to run the models and update them if necesary!
19
 
 
31
  | [mpt-7b-instruct-q4_0-ggjt.bin](https://huggingface.co/LLukas22/mpt-7b-ggml/blob/main/mpt-7b-instruct-q4_0-ggjt.bin) | [mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) | int4 | GGJT |
32
  | [mpt-7b-storywriter-f16.bin](https://huggingface.co/LLukas22/mpt-7b-ggml/blob/main/mpt-7b-f16.bin) | [mpt-7b-storywriter](https://huggingface.co/mosaicml/mpt-7b-storywriter) | fp16 | GGML |
33
  | [mpt-7b-storywriter-q4_0-ggjt.bin](https://huggingface.co/LLukas22/mpt-7b-ggml/blob/main/mpt-7b-storywriter-q4_0-ggjt.bin) | [mpt-7b-storywriter](https://huggingface.co/mosaicml/mpt-7b-storywriter) | int4 | GGJT |
34
+
35
+
36
+ ## Usage
37
+
38
+ ### Rust & Python:
39
+ #### TBD See above!
40
+
41
+ ### Via GGML
42
+ The `GGML` example only supports the ggml container type!
43
+
44
+ ##### Installation
45
+
46
+ ```
47
+ git clone https://github.com/ggerganov/ggml
48
+ cd ggml
49
+ mkdir build && cd build
50
+ cmake ..
51
+ make -j4 mpt
52
+ ```
53
+
54
+ #### Run inference
55
+
56
+ ```
57
+ ./bin/mpt -m path/to/model.bin -p "The meaning of life is"
58
+ ```