Update README.md
Browse files
README.md
CHANGED
@@ -11,6 +11,16 @@ library_name: "OpenVINO"
|
|
11 |
See the benchmark scripts in this repo.
|
12 |
|
13 |
```
|
14 |
-
pip install deepsparse-nightly==1.6.0.20231120
|
15 |
pip install openvino==2023.3.0
|
16 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
See the benchmark scripts in this repo.
|
12 |
|
13 |
```
|
14 |
+
pip install deepsparse-nightly[llm]==1.6.0.20231120
|
15 |
pip install openvino==2023.3.0
|
16 |
+
```
|
17 |
+
|
18 |
+
## Benchmarking
|
19 |
+
1. Clone this repo
|
20 |
+
2. Concatenate the big fp32 IR model:
|
21 |
+
```bash
|
22 |
+
cd ./models/neuralmagic/mpt-7b-gsm8k-pt/fp32
|
23 |
+
cat openvino_model.bin.part-a* > openvino_model.bin
|
24 |
+
```
|
25 |
+
3. Reproduce NM paper: `deepspeed_reproduce.bash`
|
26 |
+
4. OV benchmarkapp: see the files: `benchmarkapp_*.bash`, or see `run_ov_benchmark_app.bash` (more complicated).
|