Update README.md

README.md

@@ -83,17 +83,27 @@ Pre-requisite: you would need at least a machine with 4 40GB or 2 80GB NVIDIA GP
    git lfs install
    git clone https://huggingface.co/nvidia/Llama2-70B-SteerLM-Chat
    ```
-7. Run Docker container
+7. Convert checkpoint into nemo format
+   ```
+   cd Llama2-70B-SteerLM-Chat/Llama2-70B-SteerLM-Chat
+   tar -cvf Llama2-70B-SteerLM-Chat.nemo .
+   mv Llama2-70B-SteerLM-Chat.nemo ../
+   cd ..
+   # the step saves us around 130GB of space, but is optional if you have plenty of space
+   rm -r Llama2-70B-SteerLM-Chat
+   ```
+   
+9. Run Docker container
    ```
    docker run --gpus all -it --rm --shm-size=300g -p 8000:8000 -v ${PWD}/Llama2-70B-SteerLM-Chat.nemo:/opt/checkpoints/Llama2-70B-SteerLM-Chat.nemo -w /opt/NeMo nvcr.io/ea-bignlp/ga-participants/nemofw-inference:23.10
    ```
-8. Within the container, start the server in the background. This step does both conversion of the nemo checkpoint to TRT-LLM and then deployment using TRTLLM. For an explanation of each argument and advanced usage, please refer to [NeMo FW Deployment Guide](https://docs.nvidia.com/nemo-framework/user-guide/latest/deployingthenemoframeworkmodel.html)
+10. Within the container, start the server in the background. This step does both conversion of the nemo checkpoint to TRT-LLM and then deployment using TRTLLM. For an explanation of each argument and advanced usage, please refer to [NeMo FW Deployment Guide](https://docs.nvidia.com/nemo-framework/user-guide/latest/deployingthenemoframeworkmodel.html)
    
    ```
    python scripts/deploy/deploy_triton.py --nemo_checkpoint /opt/checkpoints/Llama2-70B-SteerLM-Chat.nemo --model_type="llama" --triton_model_name Llama2-70B-SteerLM-Chat --triton_http_address 0.0.0.0 --triton_port 8000 --num_gpus 2 --max_input_len 3072 --max_output_len 1024 --max_batch_size 1 &
    ```
    
-9. Once the server is ready in 20-45 mins depending on your computer (i.e. when you see this messages below), you are ready to launch your client code
+11. Once the server is ready in 20-45 mins depending on your computer (i.e. when you see this messages below), you are ready to launch your client code
 
     ```
     Started HTTPService at 0.0.0.0:8000
@@ -122,7 +132,7 @@ Pre-requisite: you would need at least a machine with 4 40GB or 2 80GB NVIDIA GP
     output = output[0][0].split("\n<extra_id_1>")[0]
     print(output)
     ```
-10. Prompt formatting for single and multi turn conversations
+12. Prompt formatting for single and multi turn conversations
     
     Single Turn
     ```