Instructions to use RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M

Use Docker

docker model run hf.co/RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M

LM Studio
Jan
Ollama
How to use RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf with Ollama:
```
ollama run hf.co/RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf with Docker Model Runner:
```
docker model run hf.co/RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M
```

Lemonade

How to use RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull RichardErkhov/Heejindo_-_rationale_model_e3_save5000_f2-gguf:Q4_K_M

Run and chat with the model

lemonade run user.Heejindo_-_rationale_model_e3_save5000_f2-gguf-Q4_K_M

List all available models

lemonade list

Atomic Chat

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Quantization made by Richard Erkhov.

Github

Discord

Request more models

rationale_model_e3_save5000_f2 - GGUF

Model creator: https://huggingface.co/Heejindo/
Original model: https://huggingface.co/Heejindo/rationale_model_e3_save5000_f2/

Name	Quant method	Size
rationale_model_e3_save5000_f2.Q2_K.gguf	Q2_K	0.54GB
rationale_model_e3_save5000_f2.IQ3_XS.gguf	IQ3_XS	0.58GB
rationale_model_e3_save5000_f2.IQ3_S.gguf	IQ3_S	0.6GB
rationale_model_e3_save5000_f2.Q3_K_S.gguf	Q3_K_S	0.6GB
rationale_model_e3_save5000_f2.IQ3_M.gguf	IQ3_M	0.61GB
rationale_model_e3_save5000_f2.Q3_K.gguf	Q3_K	0.64GB
rationale_model_e3_save5000_f2.Q3_K_M.gguf	Q3_K_M	0.64GB
rationale_model_e3_save5000_f2.Q3_K_L.gguf	Q3_K_L	0.68GB
rationale_model_e3_save5000_f2.IQ4_XS.gguf	IQ4_XS	0.7GB
rationale_model_e3_save5000_f2.Q4_0.gguf	Q4_0	0.72GB
rationale_model_e3_save5000_f2.IQ4_NL.gguf	IQ4_NL	0.72GB
rationale_model_e3_save5000_f2.Q4_K_S.gguf	Q4_K_S	0.72GB
rationale_model_e3_save5000_f2.Q4_K.gguf	Q4_K	0.75GB
rationale_model_e3_save5000_f2.Q4_K_M.gguf	Q4_K_M	0.75GB
rationale_model_e3_save5000_f2.Q4_1.gguf	Q4_1	0.77GB
rationale_model_e3_save5000_f2.Q5_0.gguf	Q5_0	0.83GB
rationale_model_e3_save5000_f2.Q5_K_S.gguf	Q5_K_S	0.83GB
rationale_model_e3_save5000_f2.Q5_K.gguf	Q5_K	0.85GB
rationale_model_e3_save5000_f2.Q5_K_M.gguf	Q5_K_M	0.85GB
rationale_model_e3_save5000_f2.Q5_1.gguf	Q5_1	0.89GB
rationale_model_e3_save5000_f2.Q6_K.gguf	Q6_K	0.95GB
rationale_model_e3_save5000_f2.Q8_0.gguf	Q8_0	1.23GB

Original model description:

library_name: transformers license: llama3.2 base_model: meta-llama/Llama-3.2-1B tags: - trl - sft - generated_from_trainer model-index: - name: rationale_model_e3_save5000_f2 results: []

rationale_model_e3_save5000_f2

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.9490

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
1.755	0.1907	1000	1.9490
1.4087	0.3815	2000	2.0212
0.9951	0.5722	3000	2.2073
0.6546	0.7629	4000	2.4321
0.3825	0.9537	5000	2.7536
0.2015	1.1444	6000	2.9396
0.1741	1.3351	7000	3.0700
0.1463	1.5258	8000	3.1767
0.1305	1.7166	9000	3.3858
0.1178	1.9073	10000	3.4989
0.0991	2.0980	11000	3.5767
0.0961	2.2888	12000	3.7036
0.095	2.4795	13000	3.8034
0.0894	2.6702	14000	3.9220
0.0862	2.8610	15000	3.9916

Framework versions

Transformers 4.45.0
Pytorch 2.3.0
Datasets 2.14.4
Tokenizers 0.20.3

Downloads last month: 1,422

GGUF

Model size

1B params

Architecture

llama

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support