tpmccallum/llama-2-13b-deep-haiku-GGML

The GGML model contained herein was generated by following the successive process in the Collab environments at https://github.com/robgon-art/DeepHaiku-LLaMa.

NOTE: This tutorial uses Meta AI's Llama-2-13b-chat-hf as the starting point (before the above quantizing process is performed). Therefore you will need to visit Meta's Llama webpage and agree to Meta's License, Acceptable Use Policy, and to Meta’s privacy policy before fetching and using Llama models.

TIP: The only change (aside from adding usernames and access tokens) was to substitute the following code (note the downgrading of llama.cpp using checkout cf348a6 below; this is so that the Collab created the older GGML version of the model instead of the newer GGUF version) in the 2_Deep_Haiku_Quantize_Model_to_GGML part of the aforementioned process:

!rm -rf llama.cpp
!git clone https://github.com/ggerganov/llama.cpp
!cd llama.cpp && git pull && git checkout cf348a6 && make clean && LLAMA_CUBLAS=1 make
!pip install numpy==1.23