Instructions to use AlexHung29629/gemma-4-E2B-sft-text-v13 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AlexHung29629/gemma-4-E2B-sft-text-v13 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="AlexHung29629/gemma-4-E2B-sft-text-v13")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("AlexHung29629/gemma-4-E2B-sft-text-v13")
model = AutoModelForMultimodalLM.from_pretrained("AlexHung29629/gemma-4-E2B-sft-text-v13", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AlexHung29629/gemma-4-E2B-sft-text-v13 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AlexHung29629/gemma-4-E2B-sft-text-v13"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlexHung29629/gemma-4-E2B-sft-text-v13",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/AlexHung29629/gemma-4-E2B-sft-text-v13

SGLang

How to use AlexHung29629/gemma-4-E2B-sft-text-v13 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AlexHung29629/gemma-4-E2B-sft-text-v13" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlexHung29629/gemma-4-E2B-sft-text-v13",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AlexHung29629/gemma-4-E2B-sft-text-v13" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlexHung29629/gemma-4-E2B-sft-text-v13",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use AlexHung29629/gemma-4-E2B-sft-text-v13 with Docker Model Runner:
```
docker model run hf.co/AlexHung29629/gemma-4-E2B-sft-text-v13
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

See axolotl config

axolotl version: 0.19.0.dev0

# Gemma-4-E2B 純文字 SFT（單機 8x H100，FSDP2)
#
# 跟 gemma4-e2b-sft.yaml（多模態,含圖片)是分開的一份設定：這份完全不吃圖片,
# 資料全部是用 prepare_pi_traces_input_output.py / fetch_nemotron_agentic.py
# 轉成的 axolotl `input_output` 格式（segments: [{label, text}, ...]),已經
# 套用過 Gemma4 chat_template 並標好哪些 special token 要訓練（每一則
# assistant 訊息在某一筆樣本裡都是「最後一則」,含它自己的 <|turn>model、
# <|channel>thought、<|tool_call>、<turn|> 等 token；tool 執行結果永遠落在
# 下一筆樣本的 label:false 前綴,不會被訓練)。
#
# 因為完全不會餵圖片，不需要 gemma4-e2b-sft.yaml 裡那些多模態專用設定
# （processor_type/skip_prepare_dataset/remove_unused_columns/image_size 等)——
# 這份走 axolotl 標準的 ChatTemplateStrategy .map() 前處理路徑，用一般
# AutoTokenizer 即可，也因此會依 sequence_len 自動丟掉過長樣本（不像
# skip_prepare_dataset: true 那條路完全不會過濾)。
#
# vision_tower/audio_tower/embed_vision/embed_audio 依然凍結
# (freeze_mm_modules)——文字資料不會用到這些參數，凍結純粹是為了省
# optimizer state，不是因為訓練圖片。

# base_model 接續 stage1（gemma4-e2b-sft-stage1-embed.yaml，embedding-only）訓練完的權重
base_model: /mnt/shared/p01/alex/E2B/outputs/gemma4-e2b-sft-text-v12-embed
hub_model_id: AlexHung29629/gemma-4-E2B-sft-text-v13

freeze_mm_modules: true

# text_config.num_hidden_layers = 35（layer index 0-34)，凍結頭尾七層
#unfrozen_parameters:
#  - model.language_model.layers.(?!(0|1|2|3|4|5|6|28|29|30|31|32|33|34).).+

# --- Plugins（跟 gemma4-e2b-sft.yaml 一致，理由同上，見該檔案註解)---
plugins:
  - axolotl.integrations.liger.LigerPlugin
  #- signsgd_plugin.SignSGDPlugin
  - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
  

#cut_cross_entropy: true

liger_rms_norm: true
liger_glu_activation: false
liger_rope: true
liger_layer_norm: false
liger_fused_linear_cross_entropy: false

strict: false
train_on_inputs: false

# --- Dataset ---
datasets:
  - path: AlexHung29629/pi_traces_sft_0807
    type: input_output
  - path: AlexHung29629/lmsys_chat_generated
    type: input_output
  - path: AlexHung29629/Qwen3.6-35B-A3B-Tool-Calling
    type: input_output


#val_set_size: 10
dataset_num_proc: 8

# --- Sequence length ---
# 六份資料合計 26493 筆，token 長度分布：nemotron_agentic 的 search 子集
# （多輪工具搜尋)尾部很長，p99 約 37k、最長 72k token；其餘資料 p99 都在
# 1.2 萬 token 內。sequence_len 拉到 32768（比 gemma4-e2b-sft.yaml 的
# 16384 大),會篩掉的過長樣本比例應會低於原本 13.4%（約 3561 筆)，
# 但尚未重新量測實際篩掉筆數——這份走標準 ChatTemplateStrategy 前處理，
# 過長樣本會被 axolotl 自動丟掉，不需要像 pi_traces 那樣另外寫腳本手動過濾。
sequence_len: 32768
pad_to_sequence_len: false
sample_packing: true
# axolotl defaults eval_causal_lm_metrics to generation-based metrics
# (sacrebleu/comet/ter/chrf)，model.generate() 對 packed 多筆 eval batch
# 會踩 out-of-bounds gather assert（跟 test_nchc_train_script_202608 的
# full.yaml 系列一致的結論），eval_strategy 有開就必須關掉 eval_sample_packing。
eval_sample_packing: false

# gemma4_hybrid_attn_impl + flash_attention_2（原設定)在 axolotl 的
# gemma4_unified monkeypatch 裡會出錯（KeyError / BlockMask 傳給 SDPA)，
# 跟 test_nchc_train_script_202608/configs/gemma4-*-full.yaml 實測結論一致：
# 只有純 flex_attention（不開 hybrid)能正常訓練。
attn_implementation: flex_attention
flex_attn_kernel_options:
  BLOCK_M: 16
  BLOCK_N: 16
  num_stages: 1
  num_warps: 4

# --- FSDP2 ---
# 注意：gemma4-e2b-sft.yaml（多模態版本）曾在這個 Gemma4 架構上遇過 FSDP2
# 第一個 forward all-gather 卡死 30 分鐘 NCCL watchdog timeout（懷疑
# auto_wrap_policy 對含 vision/audio 模組的架構切出不均勻的 FSDP unit，
# 根因未查清，見該檔案註解），因此那份改用 DDP。這份 text-only 設定雖然
# freeze_mm_modules，模型架構仍含那些模組，理論上可能踩到同一個問題——
# 先跑小 job 確認 all-gather 不會卡死。wrap class 跟
# test_nchc_train_script_202608/configs/gemma4-12b-full.yaml（同樣走
# gemma4_unified chat_template）一致。
fsdp_version: 2
fsdp_config:
  offload_params: false
  state_dict_type: SHARDED_STATE_DICT
  auto_wrap_policy: TRANSFORMER_BASED_WRAP
  transformer_layer_cls_to_wrap: Gemma4TextDecoderLayer,Gemma4VisionEncoderLayer,Gemma4VisionPatchEmbedder,Gemma4AudioLayer
  reshard_after_forward: true
  cpu_ram_efficient_loading: true
  #activation_checkpointing: true

gradient_checkpointing: true
activation_offloading: true

# --- Training ---
num_epochs: 2
micro_batch_size: 1
gradient_accumulation_steps: 8
# effective batch = 1 x 8 GPUs x 8 accum = 64 samples/step

learning_rate: 1e-5
embedding_lr_scale: 0.1
cosine_min_lr_ratio: 0.1
lr_scheduler: cosine
warmup_ratio: 0.05
weight_decay: 0.0
max_grad_norm: 1.0

adam_beta1: 0.9
adam_beta2: 0.95
adam_epsilon: 1.0e-8
optimizer: adamw_torch_8bit

bf16: true

# --- Logging / Saving / Eval ---
#eval_strategy: epoch
save_total_limit: 10
saves_per_epoch: 1
#load_best_model_at_end: true
#metric_for_best_model: eval_loss
#greater_is_better: false
logging_steps: 1

use_wandb: false
wandb_project: gemma4-e2b-sft-text
use_tensorboard: true

output_dir: ./outputs/gemma4-e2b-sft-text-v13

seed: 42
dataloader_num_workers: 8

gemma-4-E2B-sft-text-v13

This model was trained from scratch on the AlexHung29629/pi_traces_sft_0807, the AlexHung29629/lmsys_chat_generated and the AlexHung29629/Qwen3.6-35B-A3B-Tool-Calling datasets.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 8
gradient_accumulation_steps: 8
total_train_batch_size: 64
total_eval_batch_size: 8
optimizer: Use OptimizerNames.ADAMW_TORCH_8BIT with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
training_steps: 206

Training results

Framework versions

Transformers 5.14.1
Pytorch 2.11.0+cu130
Datasets 4.8.4
Tokenizers 0.22.2

Downloads last month: -

Safetensors

Model size

6B params

Tensor type

BF16

AlexHung29629
/

gemma-4-E2B-sft-text-v13

You need to agree to share your contact information to access this model

gemma-4-E2B-sft-text-v13

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Datasets used to train AlexHung29629/gemma-4-E2B-sft-text-v13