sgym-marketing-1.5b

A Qwen2.5-1.5B-Instruct model fine-tuned with Direct Preference Optimization (DPO) for marketing skill applications. Part of the SkillGym project for training LLMs to apply domain skills via reinforcement learning.

Model Description

This model is the result of the SkillGym training pipeline, which trains LLMs to apply domain-specific marketing skills. The base Qwen2.5-1.5B-Instruct model was fine-tuned using DPO on preference pairs where the "chosen" response follows a structured marketing skill template and the "rejected" response is a generic unguided output.

The model covers three marketing skill domains: CRO (Conversion Rate Optimization), Copywriting, and Cold Email outreach.

Training Details

Base model: Qwen/Qwen2.5-1.5B-Instruct
Method: Direct Preference Optimization (DPO), reference-free
LoRA config: r=4, alpha=8, dropout=0.2
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
DPO pairs: 17 (from 3 marketing skills: CRO, copywriting, cold-email)
Epochs: 10
Beta: 0.05
Training time: ~4 minutes on NVIDIA RTX 4050

Evaluation Results

Evaluated on 30 marketing tasks using a 6-dimension composite scoring system:

Dimension	Weight	Base Model	DPO Model	Delta
Criteria Pass Rate	0.25	-	-	+0.013
Specificity	0.20	-	-	+0.006
Actionability	0.20	-	-	-0.008
Structure	0.15	-	-	-0.005
Depth	0.10	-	-	-0.005
Voice	0.10	-	-	+0.054
Composite	1.00	0.311	0.318	+0.007 (+2.2%)

17/30 tasks improved
Per-skill: copywriting -0.021, cold-email +0.021, CRO +0.020
Best improvements: copy_007 (+0.180), cro_008 (+0.171), cro_009 (+0.126)

Skills Covered

CRO (Conversion Rate Optimization): Landing page analysis with heuristic evaluation, A/B test prioritization
Copywriting: Website copy creation using PAS/AIDA frameworks, brand voice matching
Cold Email: B2B outreach sequences with personalization, objection handling

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("4MLabs/sgym-marketing-1.5b")
tokenizer = AutoTokenizer.from_pretrained("4MLabs/sgym-marketing-1.5b")

messages = [
    {"role": "system", "content": "You are a marketing expert specializing in CRO."},
    {"role": "user", "content": "Analyze this landing page for conversion optimization opportunities: [landing page URL or description]"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation

If you use this model, please cite:

@article{skillgym2026,
  title={SkillGym: Training LLMs to Apply Domain Skills via Reinforcement Learning},
  author={Michiel Horstman},
  year={2026}
}