makaveli10 commited on
Commit
850e3c7
1 Parent(s): 94498e7

update README

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -12,7 +12,7 @@ Welcome to WhisperBot. WhisperBot builds upon the capabilities of the [WhisperLi
12
  Install [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/installation.md) to build Whisper and Mistral TensorRT engines. The README builds a docker image for TensorRT-LLM.
13
  Instead of building a docker image, we can also refer to the README and the [Dockerfile.multi](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docker/Dockerfile.multi) to install the required packages in the base pytroch docker image. Just make sure to use the correct base image as mentioned in the dockerfile and everything should go nice and smooth.
14
 
15
- ### Whisper
16
  - Change working dir to the [whisper example dir](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper) in TensorRT-LLM.
17
  ```bash
18
  cd TensorRT-LLM/examples/whisper
@@ -32,7 +32,7 @@ pip install -r requirements.txt
32
  python3 build.py --output_dir whisper_small_en --use_gpt_attention_plugin --use_gemm_plugin --use_layernorm_plugin --use_bert_attention_plugin --model_name small.en
33
  ```
34
 
35
- ### Mistral
36
  - Change working dir to [llama example dir](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/llama) in TensorRT-LLM folder.
37
  ```bash
38
  cd TensorRT-LLM/examples/llama
@@ -66,7 +66,7 @@ python3 main.py --whisper_tensorrt_path /root/TensorRT-LLM/examples/whisper/whis
66
  --mistral_tensorrt_path /root/TensorRT-LLM/examples/llama/tmp/mistral/7B/trt_engines/fp16/1-gpu/ \
67
  --mistral_tokenizer_path teknium/OpenHermes-2.5-Mistral-7B
68
  ```
69
- - Use the `WhisperBot/client.py` script to run on the client sidee.
70
 
71
 
72
  ## Contact Us
 
12
  Install [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/installation.md) to build Whisper and Mistral TensorRT engines. The README builds a docker image for TensorRT-LLM.
13
  Instead of building a docker image, we can also refer to the README and the [Dockerfile.multi](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docker/Dockerfile.multi) to install the required packages in the base pytroch docker image. Just make sure to use the correct base image as mentioned in the dockerfile and everything should go nice and smooth.
14
 
15
+ ### Build Whisper TensorRT Engine
16
  - Change working dir to the [whisper example dir](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper) in TensorRT-LLM.
17
  ```bash
18
  cd TensorRT-LLM/examples/whisper
 
32
  python3 build.py --output_dir whisper_small_en --use_gpt_attention_plugin --use_gemm_plugin --use_layernorm_plugin --use_bert_attention_plugin --model_name small.en
33
  ```
34
 
35
+ ### Build Mistral TensorRT Engine
36
  - Change working dir to [llama example dir](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/llama) in TensorRT-LLM folder.
37
  ```bash
38
  cd TensorRT-LLM/examples/llama
 
66
  --mistral_tensorrt_path /root/TensorRT-LLM/examples/llama/tmp/mistral/7B/trt_engines/fp16/1-gpu/ \
67
  --mistral_tokenizer_path teknium/OpenHermes-2.5-Mistral-7B
68
  ```
69
+ - Use the `WhisperBot/client.py` script to run on the client side.
70
 
71
 
72
  ## Contact Us