Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ license: llama2
|
|
11 |
|
12 |
# Llama-2 ONNX
|
13 |
|
14 |
-
This repository contains optimized version of Llama-2
|
15 |
|
16 |
## Downloading the model
|
17 |
|
@@ -25,7 +25,7 @@ pip install -U huggingface_hub
|
|
25 |
```
|
26 |
2. Download the repository.
|
27 |
```sh
|
28 |
-
huggingface-cli download alpindale/Llama-2-
|
29 |
```
|
30 |
The `--cache-dir` kwarg is only necessary if your default cache directory (`~/.cache`) does not have enough disk space to accomodate the entire repository.
|
31 |
|
@@ -33,7 +33,7 @@ The `--cache-dir` kwarg is only necessary if your default cache directory (`~/.c
|
|
33 |
The repository provides example code for running the models.
|
34 |
|
35 |
```sh
|
36 |
-
python llama2_onnx_inference.py --onnx_file FP16/
|
37 |
```
|
38 |
|
39 |
Output:
|
@@ -49,6 +49,11 @@ First, install the required packages:
|
|
49 |
pip install -r requirements.txt
|
50 |
```
|
51 |
|
|
|
|
|
|
|
|
|
|
|
52 |
Then you can simply run:
|
53 |
|
54 |
```sh
|
|
|
11 |
|
12 |
# Llama-2 ONNX
|
13 |
|
14 |
+
This repository contains optimized version of Llama-2 13B.
|
15 |
|
16 |
## Downloading the model
|
17 |
|
|
|
25 |
```
|
26 |
2. Download the repository.
|
27 |
```sh
|
28 |
+
huggingface-cli download alpindale/Llama-2-13b-ONNX --repo-type model --cache-dir /path/to/custom/cache/directory --local-dir /path/to/download/dir --local-dir-use-symlinks False
|
29 |
```
|
30 |
The `--cache-dir` kwarg is only necessary if your default cache directory (`~/.cache`) does not have enough disk space to accomodate the entire repository.
|
31 |
|
|
|
33 |
The repository provides example code for running the models.
|
34 |
|
35 |
```sh
|
36 |
+
python llama2_onnx_inference.py --onnx_file FP16/LlamaV2_13B_float16.onnx --embedding_file embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?"
|
37 |
```
|
38 |
|
39 |
Output:
|
|
|
49 |
pip install -r requirements.txt
|
50 |
```
|
51 |
|
52 |
+
Set the Python path to the root directory of the repository (necessary for importing the required modules):
|
53 |
+
```sh
|
54 |
+
export PYTHONPATH=$PYTHONPATH:$(pwd)
|
55 |
+
```
|
56 |
+
|
57 |
Then you can simply run:
|
58 |
|
59 |
```sh
|