Hazzzardous commited on
Commit
9d08f11
1 Parent(s): b716dcb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -0
README.md CHANGED
@@ -1,3 +1,26 @@
1
  ---
2
  license: mit
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
  ---
4
+
5
+ # RWKV fast quant, best used with rwkvstic 2.0.5
6
+ ## Benchmark
7
+ ### Nvidia A4000 16GB : 6.0s/100t
8
+
9
+ # Usage
10
+ ```py
11
+ from rwkvstic.load import RWKV
12
+
13
+ # Load the model (supports full path, relative path, and remote paths)
14
+
15
+ model = RWKV(
16
+ "https://huggingface.co/Hazzzardous/rwkv-fastquant/resolve/main/R14B-8K-FastQuant-rwkvstic-2-0-4.rwkv"
17
+ )
18
+
19
+ model.loadContext(newctx=f"Q: who is Jim Butcher?\n\nA:")
20
+ output = model.forward(number=100)["output"]
21
+
22
+ print(output)
23
+
24
+ # Q: who is Jim Butcher?
25
+ # A: Jim Butcher is a very popular American author of fantasy novels. He’s known for the Dresden Files series of novels.<|endoftext|>
26
+ ```