|
--- |
|
language: |
|
- en |
|
license: apache-2.0 |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- llama |
|
- gguf |
|
datasets: |
|
- yahma/alpaca-cleaned |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# Uploaded Model |
|
|
|
- **Developed by:** ar08 |
|
- **License:** apache-2.0 |
|
|
|
## USAGE |
|
|
|
To use this model, follow the steps below: |
|
|
|
1. **Install the necessary packages:** |
|
```python |
|
# Install llama-cpp-python |
|
pip install llama-cpp-python |
|
|
|
# Install transformers from source - only needed for versions <= v4.34 |
|
pip install git+https://github.com/huggingface/transformers.git |
|
|
|
# Install accelerate |
|
pip install accelerate |
|
``` |
|
|
|
2. **Instantiate the model:** |
|
```python |
|
from llama_cpp import Llama |
|
|
|
# Define the model path |
|
my_model_path = "your_downloaded_model_name/path" |
|
CONTEXT_SIZE = 512 |
|
|
|
# Load the model |
|
model = Llama(model_path=my_model_path, n_ctx=CONTEXT_SIZE) |
|
``` |
|
|
|
3. **Generate text from a prompt:** |
|
```python |
|
def generate_text_from_prompt(user_prompt, max_tokens=100, temperature=0.3, top_p=0.1, echo=True, stop=["Q", "\n"]): |
|
# Define the parameters |
|
model_output = model( |
|
user_prompt, |
|
max_tokens=max_tokens, |
|
temperature=temperature, |
|
top_p=top_p, |
|
echo=echo, |
|
stop=stop, |
|
) |
|
|
|
return model_output["choices"][0]["text"].strip() |
|
|
|
if __name__ == "__main__": |
|
my_prompt = "What do you think about the inclusion policies in Tech companies?" |
|
model_response = generate_text_from_prompt(my_prompt) |
|
print(model_response) |
|
``` |
|
|
|
|
|
``` |