yujiepan commited on
Commit
608c2fe
1 Parent(s): 7452581

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -2
README.md CHANGED
@@ -11,6 +11,16 @@ library_name: "OpenVINO"
11
  See the benchmark scripts in this repo.
12
 
13
  ```
14
- pip install deepsparse-nightly==1.6.0.20231120
15
  pip install openvino==2023.3.0
16
- ```
 
 
 
 
 
 
 
 
 
 
 
11
  See the benchmark scripts in this repo.
12
 
13
  ```
14
+ pip install deepsparse-nightly[llm]==1.6.0.20231120
15
  pip install openvino==2023.3.0
16
+ ```
17
+
18
+ ## Benchmarking
19
+ 1. Clone this repo
20
+ 2. Concatenate the big fp32 IR model:
21
+ ```bash
22
+ cd ./models/neuralmagic/mpt-7b-gsm8k-pt/fp32
23
+ cat openvino_model.bin.part-a* > openvino_model.bin
24
+ ```
25
+ 3. Reproduce NM paper: `deepspeed_reproduce.bash`
26
+ 4. OV benchmarkapp: see the files: `benchmarkapp_*.bash`, or see `run_ov_benchmark_app.bash` (more complicated).