Instructions to use daryaZare/iris-llama-3.2-3b-iris-only-k10 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use daryaZare/iris-llama-3.2-3b-iris-only-k10 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
model = PeftModel.from_pretrained(base_model, "daryaZare/iris-llama-3.2-3b-iris-only-k10")

Transformers

How to use daryaZare/iris-llama-3.2-3b-iris-only-k10 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="daryaZare/iris-llama-3.2-3b-iris-only-k10")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("daryaZare/iris-llama-3.2-3b-iris-only-k10", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use daryaZare/iris-llama-3.2-3b-iris-only-k10 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "daryaZare/iris-llama-3.2-3b-iris-only-k10"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "daryaZare/iris-llama-3.2-3b-iris-only-k10",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/daryaZare/iris-llama-3.2-3b-iris-only-k10

SGLang

How to use daryaZare/iris-llama-3.2-3b-iris-only-k10 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "daryaZare/iris-llama-3.2-3b-iris-only-k10" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "daryaZare/iris-llama-3.2-3b-iris-only-k10",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "daryaZare/iris-llama-3.2-3b-iris-only-k10" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "daryaZare/iris-llama-3.2-3b-iris-only-k10",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use daryaZare/iris-llama-3.2-3b-iris-only-k10 with Docker Model Runner:
```
docker model run hf.co/daryaZare/iris-llama-3.2-3b-iris-only-k10
```

iris-llama-3.2-3b-iris-only-k10

This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.5994
Model Preparation Time: 0.0211
Soft Mae: 0.0445
Soft Brier: 0.0070
Student Prelevantmean: 0.2169
Teacher Prelevantmean: 0.2340
Bin F1: 0.8689
Cal Ece: 0.1030
Cal Brier: 0.0609
Cal Auroc: 0.9877
Info Ndcg@p8: 0.9592
Info Pairwiseacc: 0.9264
Num Questions: 30
Num Pairs: 240

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 1
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 16
total_train_batch_size: 16
optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: constant
lr_scheduler_warmup_steps: 0.03
num_epochs: 6.0

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Soft Mae	Soft Brier	Student Prelevantmean	Teacher Prelevantmean	Bin F1	Cal Ece	Cal Brier	Cal Auroc	Info Ndcg@p8	Info Pairwiseacc	Num Questions	Num Pairs
0.7979	0.2519	34	0.6495	0.0211	0.0689	0.0153	0.2146	0.2340	0.8264	0.0876	0.0799	0.9710	0.9534	0.9271	30	240
0.7159	0.5037	68	0.5456	0.0211	0.0520	0.0093	0.2394	0.2340	0.9023	0.0830	0.0570	0.9817	0.9617	0.9314	30	240
0.7974	0.7556	102	0.6747	0.0211	0.0812	0.0137	0.2354	0.2340	0.8174	0.1493	0.0754	0.9875	0.9632	0.9429	30	240
0.6266	1.0074	136	0.4494	0.0211	0.0433	0.0063	0.2390	0.2340	0.9160	0.0936	0.0505	0.9904	0.9597	0.9299	30	240
0.6100	1.2593	170	0.5028	0.0211	0.0534	0.0100	0.2569	0.2340	0.9403	0.1267	0.0626	0.9888	0.9613	0.9298	30	240
0.4504	1.5111	204	0.4851	0.0211	0.0479	0.0074	0.2522	0.2340	0.9173	0.0958	0.0551	0.9877	0.9625	0.9330	30	240
0.5075	1.7630	238	0.5621	0.0211	0.0484	0.0087	0.2100	0.2340	0.896	0.0954	0.0553	0.9908	0.9596	0.9289	30	240
0.4004	2.0148	272	0.6911	0.0211	0.0543	0.0096	0.2037	0.2340	0.8852	0.0942	0.0653	0.9904	0.9622	0.9379	30	240
0.4990	2.2667	306	0.6354	0.0211	0.0510	0.0089	0.2485	0.2340	0.9343	0.1260	0.0645	0.9925	0.9624	0.9400	30	240
0.3352	2.5185	340	0.5159	0.0211	0.0406	0.0073	0.2445	0.2340	0.9394	0.0905	0.0489	0.9884	0.9619	0.9236	30	240
0.3843	2.7704	374	0.5069	0.0211	0.0427	0.0069	0.2214	0.2340	0.9048	0.0936	0.0569	0.9891	0.9617	0.9309	30	240
0.3380	3.0222	408	0.5225	0.0211	0.0452	0.0073	0.2361	0.2340	0.8923	0.1083	0.0595	0.9926	0.9615	0.9326	30	240
0.2921	3.2741	442	0.5340	0.0211	0.0431	0.0074	0.2437	0.2340	0.9231	0.1002	0.0583	0.9860	0.9608	0.9294	30	240
0.3617	3.5259	476	0.5901	0.0211	0.0470	0.0079	0.2140	0.2340	0.8908	0.0960	0.0617	0.9898	0.9607	0.9196	30	240
0.3146	3.7778	510	0.6594	0.0211	0.0473	0.0086	0.2543	0.2340	0.9275	0.0998	0.0549	0.9902	0.9636	0.9425	30	240
0.4398	4.0296	544	0.6077	0.0211	0.0506	0.0074	0.2421	0.2340	0.9333	0.1214	0.0602	0.9924	0.9615	0.9318	30	240
0.4297	4.2815	578	0.7119	0.0211	0.0536	0.0093	0.2112	0.2340	0.8739	0.1116	0.0689	0.9914	0.9618	0.9331	30	240
0.2771	4.5333	612	0.7199	0.0211	0.0469	0.0072	0.2187	0.2340	0.8926	0.1141	0.0620	0.9930	0.9611	0.9322	30	240
0.2916	4.7852	646	0.5541	0.0211	0.0431	0.0070	0.2240	0.2340	0.9032	0.0929	0.0551	0.9909	0.9620	0.9337	30	240
0.2044	5.0370	680	0.6737	0.0211	0.0458	0.0074	0.2241	0.2340	0.8780	0.0950	0.0599	0.9889	0.9609	0.9315	30	240
0.2504	5.2889	714	0.8273	0.0211	0.0585	0.0093	0.2307	0.2340	0.8943	0.1379	0.0649	0.9937	0.9621	0.9331	30	240
0.2824	5.5407	748	0.6957	0.0211	0.0509	0.0079	0.2325	0.2340	0.9302	0.1271	0.0624	0.9924	0.9611	0.9295	30	240
0.3054	5.7926	782	0.5287	0.0211	0.0434	0.0072	0.2465	0.2340	0.9265	0.0990	0.0552	0.9895	0.9626	0.9213	30	240
0.2570	6.0	810	0.5994	0.0211	0.0445	0.0070	0.2169	0.2340	0.8689	0.1030	0.0609	0.9877	0.9592	0.9264	30	240

Framework versions

PEFT 0.19.1
Transformers 5.14.1
Pytorch 2.5.1+cu124
Datasets 5.0.0
Tokenizers 0.22.2

Downloads last month: -

Model tree for daryaZare/iris-llama-3.2-3b-iris-only-k10

Base model

meta-llama/Llama-3.2-3B-Instruct

Adapter

(790)

this model