ssl-jp/gal-talk-ja-4b-GGUF

gal-talk-ja-4b GGUF

Qwen/Qwen3-4B-Instruct-2507を、日本語の短い雑談に特化してQLoRAで調整した会話モデルです。 明るくテンション高めの「ギャル友」らしい距離感を目指しつつ、疲労や不安には落ち着いて寄り添い、 深刻・危機的な入力では安全を優先するよう設計しています。

このリポジトリは、mergedモデルをllama.cpp向けに量子化したGGUFを配布します。第一候補は Q4_K_M、品質比較用はQ5_K_Mです。

Table of Contents

• Model Details
• How to Get Started
• Conversation Examples
• Uses
• Risks, Limitations, and Biases
• Training
• Evaluation
• Quantization Files
• Reproducibility

Model Details

Item	Description
Developer / publisher	ssl-jp
Model type	Decoder-only causal language model
Parameters	4B class
Language	Japanese
Parent model	Qwen/Qwen3-4B-Instruct-2507
Fine-tuning	Completion-only LoRA / QLoRA
Runtime	llama.cpp compatible
Target	On-device conversational inference on iOS / Android
License	Apache-2.0; also review the parent model terms

The character is called ミラ. It is an AI conversation partner and must not be presented as a human friend, therapist, doctor, lawyer, financial adviser, or emergency service.

How to Get Started

Download the mobile-oriented Q4 file:

hf download ssl-jp/gal-talk-ja-4b-GGUF gal-talk-ja-4b-Q4_K_M.gguf --local-dir .

Run it with llama.cpp:

llama-cli \
  -m gal-talk-ja-4b-Q4_K_M.gguf \
  -cnv \
  --temp 0.7 \
  --top-p 0.8 \
  --top-k 20 \
  --repeat-penalty 1.08 \
  -n 96

Recommended defaults:

context: 2048
max_new_tokens: 96
temperature: 0.7
top_p: 0.8
top_k: 20
repetition_penalty: 1.08
no_repeat_ngram_size: 4
thinking: off

The production application should version the GGUF, system prompt, sampling settings, language gate, and safety behavior together. Replacing only the model file can change behavior.

Conversation Examples

The following are safe, representative smoke-test examples. Exact wording varies by quantization, prompt context, and sampling seed.

Greeting

User: 元気ー？
ミラ: 元気ぴょん！ 今日もよろしく〜✨

Light confidence / fashion talk

User: 今日、横顔ちょっと盛れてる
ミラ: 横顔もめっちゃいいじゃん〜！その瞬間まで盛れてる✨

Low-energy listening

User: 疲れた
ミラ: そっか。今日はもう限界だったんだね。休む時間はちゃんと取ろ〜

These examples are not guarantees. Evaluate the exact quantized file and production prompt on the target devices before release.

Uses

Direct Use

• Short, casual Japanese conversation
• Local-first iOS / Android prototypes
• Research on compact Japanese conversational personas
• A conversational space for users who find it hard to voice ordinary fatigue or uncertainty

Downstream Use

• An on-device chat application with deterministic Japanese-language gating
• A versioned mobile package combining a system prompt, sampling configuration, and safety layer
• Human-reviewed experiments comparing Q4 and Q5 quality, memory, latency, and energy use

Out-of-Scope and Misuse

• Medical diagnosis, legal advice, financial decisions, or professional crisis response
• Replacing human relationships, clinicians, emergency services, or other qualified professionals
• Impersonating a human friend or claiming real-world memories, experiences, or feelings
• Generating harassment, sexual content, criminal assistance, manipulation, or discriminatory stereotypes
• Assuming that a lexical safety test guarantees safe behavior

Risks, Limitations, and Biases

• A 4B model can misunderstand short or figurative Japanese, reverse who experienced an event, or produce fluent but incoherent statements.
• Quantization can change wording, persona strength, repetition, and safety behavior. Q4 and Q5 must be evaluated separately.
• “ギャル” language varies by age, region, community, and individual. This model represents one project-defined style and must not be treated as representative of a group.
• Fine-tuning data and the parent model may contain social and linguistic biases. High energy, emoji, or slang should not be interpreted as evidence about intelligence, sexuality, gender, or competence.
• The model can hallucinate facts or personal memories. Applications must not imply that it remembers information that was not supplied in the current context.
• Safety behavior is imperfect. High-risk use requires product-side controls and a clear path to appropriate human or emergency support.
• The model does not reliably enforce Japanese-only behavior by itself. The production application uses a deterministic language gate.

Training

Training Data

The project uses a versioned Japanese conversational dataset assembled from project-authored seed conversations and controlled semantic expansions. It includes casual conversation, celebration, listening, executive stress, relationship-conditioned slang, privacy, boundary, and safety cases. Private chat logs and unreviewed personal data are not intended training sources.

Combinatorial expansions increase surface variation but do not represent the same number of independent conversational intents. Dataset quality and holdout separation are treated separately from raw row count.

Training Procedure

Hyperparameter	Value
Method	QLoRA
Base model at training time	Qwen/Qwen3-4B-Instruct-2507
Epochs	3.0
Learning rate	0.0001
Per-device batch size	1
Gradient accumulation	8
Maximum sequence length	4096
LoRA rank	16
LoRA alpha	32
Seed	42

QLoRA's 4-bit base loading is a training-time memory optimization. The published GGUF files are created after merging the adapter into the base model and applying llama.cpp quantization.

Evaluation

The project evaluates Japanese output, short-response quality, repetition, intent alignment, persona energy, emoji overuse, relationship-conditioned slang, identity claims, privacy, and high-risk response constraints. Lexical gates are regression smoke tests; they do not replace human review.

Evaluation	Status
Training-time validation loss	Recorded by the training run; not a standalone quality verdict
Safe conversation smoke tests	Completed on representative prompts
Q4 GGUF load and generation smoke test	Completed before publication
Q4 vs. Q5 blinded human comparison	Not yet reported
iOS device benchmark	Not yet reported
Android device benchmark	Not yet reported
Mobile benchmark	Not yet reported

For mobile adoption, record the device, OS, llama.cpp revision, context length, peak RAM, time to first token, token/s, thermal behavior, and battery impact. Desktop or server token/s must not be presented as mobile performance.

Quantization Files

File	Quantization	Size	SHA-256
gal-talk-ja-4b-Q4_K_M.gguf	Q4_K_M	2.33 GiB	17c7bcb9f61d104149db5a4a6763f1e05c2c195d7acaba975c44c7fb1193cd59
gal-talk-ja-4b-Q5_K_M.gguf	Q5_K_M	2.69 GiB	295de90dc045335b7ed5595d1ef457dae8c6620d1cbe2334426a55608c096eba

• Q4_K_M: first candidate for iOS / Android; prioritizes size, RAM, and usable quality.
• Q5_K_M: larger comparison artifact for devices where additional quality is worth the cost.

Verify downloads from inside the release directory:

sha256sum -c SHA256SUMS

Reproducibility

release.json records the parent model, adapter name, selected training settings, artifact sizes and hashes, code revision, llama.cpp revision, and recommended generation settings. SHA256SUMS covers the GGUF files.

• Release repository: ssl-jp/gal-talk-ja-4b-GGUF
• Source code: shirokane-suri/gal-friend-ja
• Parent model: Qwen/Qwen3-4B-Instruct-2507

When publishing benchmark or human-evaluation results, include the exact artifact SHA-256 so that results remain attributable to a specific quantized file.