metadata

language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - llama
  - gguf
datasets:
  - yahma/alpaca-cleaned
library_name: transformers
pipeline_tag: text-generation

Uploaded Model

Developed by: ar08
License: apache-2.0

USAGE

To use this model, follow the steps below:

Install the necessary packages:

# Install llama-cpp-python
pip install llama-cpp-python

# Install transformers from source - only needed for versions <= v4.34
pip install git+https://github.com/huggingface/transformers.git

# Install accelerate
pip install accelerate

Instantiate the model:

from llama_cpp import Llama

# Define the model path
my_model_path = "your_downloaded_model_name/path"
CONTEXT_SIZE = 512

# Load the model
model = Llama(model_path=my_model_path, n_ctx=CONTEXT_SIZE)

Generate text from a prompt:

def generate_text_from_prompt(user_prompt, max_tokens=100, temperature=0.3, top_p=0.1, echo=True, stop=["Q", "\n"]):
    # Define the parameters
    model_output = model(
        user_prompt,
        max_tokens=max_tokens,
        temperature=temperature,
        top_p=top_p,
        echo=echo,
        stop=stop,
    )

    return model_output["choices"][0]["text"].strip()

if __name__ == "__main__":
    my_prompt = "What do you think about the inclusion policies in Tech companies?"
    model_response = generate_text_from_prompt(my_prompt)
    print(model_response)