Instructions to use winstonlin800/openllama3b-alpaca with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use winstonlin800/openllama3b-alpaca with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="winstonlin800/openllama3b-alpaca")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("winstonlin800/openllama3b-alpaca")
model = AutoModelForCausalLM.from_pretrained("winstonlin800/openllama3b-alpaca", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use winstonlin800/openllama3b-alpaca with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "winstonlin800/openllama3b-alpaca"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "winstonlin800/openllama3b-alpaca",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/winstonlin800/openllama3b-alpaca

SGLang

How to use winstonlin800/openllama3b-alpaca with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "winstonlin800/openllama3b-alpaca" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "winstonlin800/openllama3b-alpaca",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "winstonlin800/openllama3b-alpaca" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "winstonlin800/openllama3b-alpaca",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use winstonlin800/openllama3b-alpaca with Docker Model Runner:
```
docker model run hf.co/winstonlin800/openllama3b-alpaca
```

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Fine-tuned Open LLaMA-3B on Code Alpaca This repository hosts the Open LLaMA 3B model, fine-tuned on the Code Alpaca dataset. Our goal is to enhance the original LLaMA model's capabilities in understanding and generating code using small but robust enough to run on consumer pc, making it a powerful tool for developers, researchers, and hobbyists looking to leverage AI for software development tasks.

Model Description The Open LLaMA 3B model, initially developed by Meta AI, is a state-of-the-art language model known for its impressive natural language understanding and generation capabilities. By fine-tuning this model on the Code Alpaca dataset, we've tailored it to better comprehend and produce code, significantly reducing the gap between AI and practical programming tasks.

Features

** Enhanced Code Generation: Generates syntactically correct and logically coherent code snippets. ** Multi-Language Support: Understands and generates code in multiple programming languages, including but not limited to Python, JavaScript, and Java. ** Error Reduction: Produces code with fewer errors, minimizing the need for manual debugging. ** Domain-Specific Knowledge: Adapts to specific programming frameworks and libraries, making it useful for a wide range of software development projects.

Installation

To install and use the fine-tuned Open LLaMA 3B model, please follow these steps:

bash Copy code pip install transformers Then, load the model using the Hugging Face transformers library:

python Copy code from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "your_hf_username/fine_tuned_open_llama_3b_code_alpaca" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

Usage

Here's a quick example of how to generate code using the fine-tuned model:

python

Copy code prompt = "Write a Python function to add two numbers:" inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate(inputs["input_ids"], max_length=50) print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Performance -- to be added

Limitations and Bias -- to be added

How to Contribute We welcome contributions to improve the model's performance and usability! Please refer to the contributing guidelines for more information on submitting pull requests or reporting issues.

License Apache 2.0

Acknowledgments Special thanks to the creators of the Open LLaMA model at Meta AI and the contributors to the Code Alpaca dataset. Their work has made this fine-tuned model possible.

Downloads last month: 14