Instructions to use AlexHung29629/gemma-4-E2B-sft-stage1-embed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AlexHung29629/gemma-4-E2B-sft-stage1-embed with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="AlexHung29629/gemma-4-E2B-sft-stage1-embed")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("AlexHung29629/gemma-4-E2B-sft-stage1-embed")
model = AutoModelForMultimodalLM.from_pretrained("AlexHung29629/gemma-4-E2B-sft-stage1-embed", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AlexHung29629/gemma-4-E2B-sft-stage1-embed with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AlexHung29629/gemma-4-E2B-sft-stage1-embed"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlexHung29629/gemma-4-E2B-sft-stage1-embed",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/AlexHung29629/gemma-4-E2B-sft-stage1-embed

SGLang

How to use AlexHung29629/gemma-4-E2B-sft-stage1-embed with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AlexHung29629/gemma-4-E2B-sft-stage1-embed" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlexHung29629/gemma-4-E2B-sft-stage1-embed",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AlexHung29629/gemma-4-E2B-sft-stage1-embed" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlexHung29629/gemma-4-E2B-sft-stage1-embed",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use AlexHung29629/gemma-4-E2B-sft-stage1-embed with Docker Model Runner:
```
docker model run hf.co/AlexHung29629/gemma-4-E2B-sft-stage1-embed
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

See axolotl config

axolotl version: 0.18.0

# Gemma-4-E2B SFT — Stage 1／2：只訓練 input/output embedding（tie_word_embeddings=true，
# 兩者共用同一份 model.language_model.embed_tokens.weight），其餘全部凍結。
# 目的：讓新 chat_template 特殊符號 / 新增的 system-role 訓練分佈先在 embedding
# 層對齊，再進入 stage 2（gemma4-e2b-sft.yaml）做全參數訓練，降低一開始全參數
# 更新對預訓練權重造成的擾動。
#
# unfrozen_parameters 是唯一控制凍結的機制（axolotl utils/freeze.freeze_layers_except）：
# 沒列到的參數全部 requires_grad=False，freeze_mm_modules 對 vision/audio 的凍結
# 因此變成多餘但無害。model.language_model.embed_tokens_per_layer.weight 是 Gemma4
# 專屬的 per-layer embedding（altup/laurel 用），不是一般認知的 input/output embedding，
# 故不列入。
#
# roles_to_train 涵蓋全部角色（system/user/assistant/tool）：這個階段只是讓
# embedding 對齊全部 vocab（含 user/tool 訊息裡才會出現的 token），loss 覆蓋
# 全部 token 才能讓每個 token 的 embedding 都拿到梯度；stage2（僅
# system/assistant）才回到正常的「只在模型該生成的內容上計算 loss」設定。
#
# 其餘設定（dataset/plugins/sequence_len/...）與 stage2（gemma4-e2b-sft.yaml）保持一致，
# 只覆寫本檔案列出的欄位；若 stage2 的共用設定調整，記得同步這裡。

base_model: AlexHung29629/gemma-4-E2B
hub_model_id: AlexHung29629/gemma-4-E2B-sft-stage1-embed

# --- Multimodal ---
processor_type: AutoProcessor
skip_prepare_dataset: true
remove_unused_columns: false
sample_packing: false

freeze_mm_modules: true

# --- Freeze 除 embedding 以外的所有參數 ---
unfrozen_parameters:
  - ^model.language_model.embed_tokens.weight$
  - ^lm_head.weight$

# --- Plugins ---
plugins:
  - axolotl.integrations.liger.LigerPlugin
  - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin

cut_cross_entropy: true

liger_rms_norm: true
liger_glu_activation: false
liger_rope: true
liger_layer_norm: false
liger_fused_linear_cross_entropy: false

strict: false

# --- Dataset ---
datasets:
  - path: data/pi_traces_sft.jsonl
    type: chat_template
    field_messages: messages
    field_tools: tools
    split: train
    message_property_mappings:
      role: role
      content: content
      tool_calls: tool_calls
      tool_call_id: tool_call_id
      name: name
      reasoning_content: reasoning_content

    train_on_eos: all
    train_on_eot: all
    # 全部角色都計算 loss（與 stage2 唯一的 dataset 差異）
    roles_to_train:
      - system
      - user
      - assistant
      - tool

chat_template_kwargs:
  enable_thinking: true
  preserve_thinking: true

val_set_size: 16
dataset_num_proc: 8

image_size: 512
image_resize_algorithm: bilinear

sequence_len: 16384
pad_to_sequence_len: false

attn_implementation: flash_attention_2
gemma4_hybrid_attn_impl: true

gradient_checkpointing: true

# --- Training ---
# 只訓練 embedding，收斂快、成本低，epoch 數遠少於後續全參數訓練。
num_epochs: 1
micro_batch_size: 1
gradient_accumulation_steps: 8

learning_rate: 1.0e-5
lr_scheduler: constant_with_warmup
warmup_ratio: 0.03
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.95
adam_epsilon: 1.0e-8
max_grad_norm: 1.0
optimizer: adamw_torch_8bit

bf16: true

eval_strategy: epoch
save_strategy: best
save_total_limit: 1
load_best_model_at_end: true
metric_for_best_model: eval_loss
greater_is_better: false
logging_steps: 5

use_wandb: true
wandb_project: gemma4-e2b-sft
wandb_run_id: stage1-embed
use_tensorboard: true

output_dir: ./outputs/gemma4-e2b-sft-stage1-embed

seed: 42
dataloader_num_workers: 8

gemma-4-E2B-sft-stage1-embed

This model is a fine-tuned version of AlexHung29629/gemma-4-E2B on the data/pi_traces_sft.jsonl dataset. It achieves the following results on the evaluation set:

Loss: 2.6218
Ppl: 13.7603
Memory/max Active (gib): 12.49
Memory/max Allocated (gib): 12.49
Memory/device Reserved (gib): 17.75

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 8
gradient_accumulation_steps: 8
total_train_batch_size: 64
total_eval_batch_size: 8
optimizer: Use OptimizerNames.ADAMW_TORCH_8BIT with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: constant_with_warmup
training_steps: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Ppl	Active (gib)	Allocated (gib)	Reserved (gib)
No log	0	0	2.6269	13.8310	14.29	14.29	19.78
2.5937	1.0	10	2.6218	13.7603	12.49	12.49	17.75

Framework versions

Transformers 5.14.1
Pytorch 2.11.0
Datasets 4.8.4
Tokenizers 0.22.2

Downloads last month: 21

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for AlexHung29629/gemma-4-E2B-sft-stage1-embed

Base model

AlexHung29629/gemma-4-E2B

Finetuned

(2)

this model

Finetunes

1 model