Instructions to use ueihieu/llama-2-7b-genwiki-context with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ueihieu/llama-2-7b-genwiki-context with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("NousResearch/Llama-2-7b-hf")
model = PeftModel.from_pretrained(base_model, "ueihieu/llama-2-7b-genwiki-context")

Transformers

How to use ueihieu/llama-2-7b-genwiki-context with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ueihieu/llama-2-7b-genwiki-context")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ueihieu/llama-2-7b-genwiki-context", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ueihieu/llama-2-7b-genwiki-context with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ueihieu/llama-2-7b-genwiki-context"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ueihieu/llama-2-7b-genwiki-context",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ueihieu/llama-2-7b-genwiki-context

SGLang

How to use ueihieu/llama-2-7b-genwiki-context with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ueihieu/llama-2-7b-genwiki-context" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ueihieu/llama-2-7b-genwiki-context",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ueihieu/llama-2-7b-genwiki-context" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ueihieu/llama-2-7b-genwiki-context",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ueihieu/llama-2-7b-genwiki-context with Docker Model Runner:
```
docker model run hf.co/ueihieu/llama-2-7b-genwiki-context
```

llama-2-7b-genwiki-context

This model is a fine-tuned version of NousResearch/Llama-2-7b-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 9.4690

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 32
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
training_steps: 500
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
9.9820	0.0008	20	9.9401
9.7511	0.0016	40	9.7085
9.5427	0.0024	60	9.5100
9.4889	0.0032	80	9.4915
9.5876	0.0040	100	9.4849
9.3745	0.0048	120	9.4810
9.5359	0.0056	140	9.4787
9.4569	0.0064	160	9.4769
9.5698	0.0072	180	9.4757
9.5392	0.0080	200	9.4748
9.4891	0.0088	220	9.4739
9.4572	0.0096	240	9.4728
9.5333	0.0104	260	9.4729
9.4396	0.0112	280	9.4718
9.3508	0.0119	300	9.4716
9.5687	0.0127	320	9.4711
9.5233	0.0135	340	9.4707
9.4928	0.0143	360	9.4704
9.5720	0.0151	380	9.4700
9.6364	0.0159	400	9.4698
9.5459	0.0167	420	9.4696
9.5185	0.0175	440	9.4694
9.5466	0.0183	460	9.4693
9.6462	0.0191	480	9.4691
9.5411	0.0199	500	9.4690

Framework versions

PEFT 0.20.0
Transformers 5.0.0
Pytorch 2.10.0+cu128
Datasets 5.0.0
Tokenizers 0.22.2

Downloads last month: 386

Model tree for ueihieu/llama-2-7b-genwiki-context

Base model

NousResearch/Llama-2-7b-hf

Adapter

(139)

this model