metadata
base_model:
- t-tech/T-lite-it-1.0
Q4_K_M vestion of https://huggingface.co/t-tech/T-lite-it-1.0
Llama-cpp-python code:
from llama_cpp import Llama
from huggingface_hub import snapshot_download
# load model
snapshot_download(repo_id="ichrnkv/t_lite_1.0_gguf", local_dir="./")
# llama cpp model
model = Llama(
model_path="./model.gguf",
verbose=True,
n_gpu_layers=-1,
seed=42
)