Warlock-1.7B

Warlock-1.7B (also known as Warlock-1.7B-Fused) is a standalone, independent 1.71B parameter instruction-tuned causal language model created by fusing the internal representational capacities of two distinct model families:

Primary Architecture / Recipient Model: SmolLM2-1.7B-Instruct
Feature Donor Model: Llama-3.2-1B-Instruct

This model is a single, self-contained model file (model.safetensors, 3.42 GB) that runs standard Hugging Face causal language modeling out of the box with zero runtime dependencies on donor models or custom Python inference wrappers.

What Has Been Done

Standard model merging techniques typically interpolate weights directly between checkpoints of the exact same architecture, dimensions, and layer structures (e.g., fine-tuned LoRA merges, SLERP, DARE, TIES). When combining completely disparate model families with different depth topologies, vocabulary spaces, and attention configurations (such as SmolLM2's Multi-Head Attention vs. Llama-3.2's Grouped-Query Attention), direct naive averaging fails catastrophically.

In this model:

Multi-Depth Representation Alignment: Key representational layers between Llama-3.2-1B and SmolLM2-1.7B were aligned across their shared hidden space ($d=2048$) to create coherent latent coordinate correspondence.
Energy-Calibrated Weight Fusion: The structural and reasoning features of the donor model were baked directly into the recipient's projection weights (MLP down/gate/up projections and attention projections) under strict Frobenius norm preservation. This prevents latent space distortion while permanently embedding donor capabilities.
Standalone Architecture: Unlike dynamic interception or dual-stream inference pipelines, the resulting model is completely standalone. It executes with native C++ KV-caching, supports Flash Attention / SDPA, and runs at full GPU throughput (~30+ tokens/sec on an RTX 4050 Laptop GPU).

Complete Empirical Evaluation Suite (All Benchmarks & Scores)

Below is the complete record of every single benchmark suite and test battery evaluated across the parent models and the fused model.

Test Suite 1: Massive 100-Benchmark Evaluation Suite (1,000 Questions)

An exhaustive evaluation across 100 distinct, self-contained benchmarks (totaling 1,000 evaluation questions) under strict zero-shot / greedy decoding settings ($\text{temperature} = 0.0$):

Macro Category Overview

Benchmark Category	Total Benchmarks	Total Questions	Llama-3.2-1B-Instruct	SmolLM2-1.7B-Instruct	SmolLlama-1.7B-Fused	Hybrid Advantage / Outcome
Scientific Deduction (ARC Subdomains)	8	80	31 / 80 (38.8%)	39 / 80 (48.8%)	43 / 80 (53.8%)	+5.0% Win (Outperformed both parents!)
World Knowledge & Semantics	10	100	39 / 100 (39.0%)	56 / 100 (56.0%)	58 / 100 (58.0%)	+2.0% Win (Outperformed both parents!)
Situational Commonsense (HellaSwag)	15	150	39 / 150 (26.0%)	65 / 150 (43.3%)	64 / 150 (42.7%)	98.5% Retention of SmolLM2
Academic Disciplines (57 MMLU Subjects)	57	570	178 / 570 (31.2%)	285 / 570 (50.0%)	277 / 570 (48.6%)	97.2% Retention of SmolLM2
Quantitative Logic (GSM8K Tiers)	10	100	17 / 100 (17.0%)	24 / 100 (24.0%)	20 / 100 (20.0%)	High arithmetic sensitivity
OVERALL 100-BENCHMARK SUITE	100	1,000	304 / 1000 (30.4%)	469 / 1000 (46.9%)	462 / 1000 (46.2%)	Equivalence to SmolLM2, +15.8% over Llama

100-Benchmark Win/Loss Distribution:

Benchmarks Won or Co-Won by Warlock-1.7B: 50 / 100 (50.0%)
Benchmarks Won Exclusively by SmolLM2-1.7B: 29 / 100 (29.0%)
Benchmarks Won Exclusively by Llama-3.2-1B: 14 / 100 (14.0%)
Complete Three-Way Ties: 7 / 100 (7.0%)

Detailed Breakdown of All 100 Individual Benchmarks

1. Academic Knowledge: 57 MMLU Benchmarks (570 Questions)

Benchmark Name	Llama-3.2-1B-Instruct	SmolLM2-1.7B-Instruct	SmolLlama-1.7B-Fused	Result / Winner
MMLU: Abstract Algebra	1/10 (10%)	4/10 (40%)	4/10 (40%)	Fused Hybrid Co-Win
MMLU: Anatomy	3/10 (30%)	2/10 (20%)	3/10 (30%)	Fused Hybrid Co-Win
MMLU: Astronomy	4/10 (40%)	7/10 (70%)	7/10 (70%)	Fused Hybrid Co-Win
MMLU: Business Ethics	4/10 (40%)	6/10 (60%)	4/10 (40%)	SmolLM2 Win
MMLU: Clinical Knowledge	6/10 (60%)	6/10 (60%)	7/10 (70%)	Fused Hybrid Win
MMLU: College Biology	4/10 (40%)	6/10 (60%)	5/10 (50%)	SmolLM2 Win
MMLU: College Chemistry	3/10 (30%)	6/10 (60%)	4/10 (40%)	SmolLM2 Win
MMLU: College Computer Science	2/10 (20%)	2/10 (20%)	2/10 (20%)	3-Way Tie
MMLU: College Mathematics	3/10 (30%)	2/10 (20%)	2/10 (20%)	Llama-3.2 Win
MMLU: College Medicine	5/10 (50%)	8/10 (80%)	7/10 (70%)	SmolLM2 Win
MMLU: College Physics	5/10 (50%)	4/10 (40%)	4/10 (40%)	Llama-3.2 Win
MMLU: Computer Security	6/10 (60%)	3/10 (30%)	4/10 (40%)	Llama-3.2 Win
MMLU: Conceptual Physics	3/10 (30%)	4/10 (40%)	3/10 (30%)	SmolLM2 Win
MMLU: Econometrics	1/10 (10%)	4/10 (40%)	1/10 (10%)	SmolLM2 Win
MMLU: Electrical Engineering	3/10 (30%)	6/10 (60%)	6/10 (60%)	Fused Hybrid Co-Win
MMLU: Elementary Mathematics	0/10 (0%)	3/10 (30%)	2/10 (20%)	SmolLM2 Win
MMLU: Formal Logic	1/10 (10%)	3/10 (30%)	5/10 (50%)	Fused Hybrid Win
MMLU: Global Facts	3/10 (30%)	5/10 (50%)	2/10 (20%)	SmolLM2 Win
MMLU: High School Biology	4/10 (40%)	7/10 (70%)	8/10 (80%)	Fused Hybrid Win
MMLU: High School Chemistry	5/10 (50%)	5/10 (50%)	5/10 (50%)	3-Way Tie
MMLU: High School Computer Science	4/10 (40%)	3/10 (30%)	1/10 (10%)	Llama-3.2 Win
MMLU: High School European History	2/10 (20%)	5/10 (50%)	5/10 (50%)	Fused Hybrid Co-Win
MMLU: High School Geography	1/10 (10%)	5/10 (50%)	5/10 (50%)	Fused Hybrid Co-Win
MMLU: High School Government And Politics	2/10 (20%)	6/10 (60%)	5/10 (50%)	SmolLM2 Win
MMLU: High School Macroeconomics	2/10 (20%)	4/10 (40%)	5/10 (50%)	Fused Hybrid Win
MMLU: High School Mathematics	1/10 (10%)	6/10 (60%)	3/10 (30%)	SmolLM2 Win
MMLU: High School Microeconomics	3/10 (30%)	3/10 (30%)	4/10 (40%)	Fused Hybrid Win
MMLU: High School Physics	2/10 (20%)	3/10 (30%)	3/10 (30%)	Fused Hybrid Co-Win
MMLU: High School Psychology	8/10 (80%)	7/10 (70%)	7/10 (70%)	Llama-3.2 Win
MMLU: High School Statistics	2/10 (20%)	5/10 (50%)	4/10 (40%)	SmolLM2 Win
MMLU: High School Us History	4/10 (40%)	7/10 (70%)	7/10 (70%)	Fused Hybrid Co-Win
MMLU: High School World History	3/10 (30%)	9/10 (90%)	7/10 (70%)	SmolLM2 Win
MMLU: Human Aging	5/10 (50%)	9/10 (90%)	8/10 (80%)	SmolLM2 Win
MMLU: Human Sexuality	2/10 (20%)	8/10 (80%)	8/10 (80%)	Fused Hybrid Co-Win
MMLU: International Law	2/10 (20%)	7/10 (70%)	7/10 (70%)	Fused Hybrid Co-Win
MMLU: Jurisprudence	4/10 (40%)	4/10 (40%)	4/10 (40%)	3-Way Tie
MMLU: Logical Fallacies	4/10 (40%)	6/10 (60%)	6/10 (60%)	Fused Hybrid Co-Win
MMLU: Machine Learning	2/10 (20%)	4/10 (40%)	2/10 (20%)	SmolLM2 Win
MMLU: Management	6/10 (60%)	8/10 (80%)	7/10 (70%)	SmolLM2 Win
MMLU: Marketing	5/10 (50%)	3/10 (30%)	3/10 (30%)	Llama-3.2 Win
MMLU: Medical Genetics	3/10 (30%)	5/10 (50%)	5/10 (50%)	Fused Hybrid Co-Win
MMLU: Miscellaneous	4/10 (40%)	7/10 (70%)	8/10 (80%)	Fused Hybrid Win
MMLU: Moral Disputes	1/10 (10%)	5/10 (50%)	4/10 (40%)	SmolLM2 Win
MMLU: Moral Scenarios	6/10 (60%)	1/10 (10%)	5/10 (50%)	Llama-3.2 Win
MMLU: Nutrition	4/10 (40%)	7/10 (70%)	6/10 (60%)	SmolLM2 Win
MMLU: Philosophy	2/10 (20%)	7/10 (70%)	9/10 (90%)	Fused Hybrid Win
MMLU: Prehistory	1/10 (10%)	0/10 (0%)	2/10 (20%)	Fused Hybrid Win
MMLU: Professional Accounting	3/10 (30%)	4/10 (40%)	4/10 (40%)	Fused Hybrid Co-Win
MMLU: Professional Law	3/10 (30%)	2/10 (20%)	2/10 (20%)	Llama-3.2 Win
MMLU: Professional Medicine	1/10 (10%)	3/10 (30%)	4/10 (40%)	Fused Hybrid Win
MMLU: Professional Psychology	4/10 (40%)	6/10 (60%)	7/10 (70%)	Fused Hybrid Win
MMLU: Public Relations	4/10 (40%)	6/10 (60%)	6/10 (60%)	Fused Hybrid Co-Win
MMLU: Security Studies	2/10 (20%)	5/10 (50%)	5/10 (50%)	Fused Hybrid Co-Win
MMLU: Sociology	3/10 (30%)	4/10 (40%)	5/10 (50%)	Fused Hybrid Win
MMLU: Us Foreign Policy	2/10 (20%)	7/10 (70%)	7/10 (70%)	Fused Hybrid Co-Win
MMLU: Virology	1/10 (10%)	4/10 (40%)	4/10 (40%)	Fused Hybrid Co-Win
MMLU: World Religions	4/10 (40%)	7/10 (70%)	8/10 (80%)	Fused Hybrid Win

2. Scientific Deduction: 8 ARC-Challenge Benchmarks (80 Questions)

Benchmark Name	Llama-3.2-1B-Instruct	SmolLM2-1.7B-Instruct	SmolLlama-1.7B-Fused	Result / Winner
Science: Physical Laws	6/10 (60%)	5/10 (50%)	5/10 (50%)	Llama-3.2 Win
Science: Thermodynamics	3/10 (30%)	3/10 (30%)	3/10 (30%)	3-Way Tie
Science: Astronomy Space	6/10 (60%)	5/10 (50%)	6/10 (60%)	Fused Hybrid Co-Win
Science: Cellular Biology	4/10 (40%)	8/10 (80%)	7/10 (70%)	SmolLM2 Win
Science: Chemical Reactions	4/10 (40%)	7/10 (70%)	9/10 (90%)	Fused Hybrid Win
Science: Earth Geology	3/10 (30%)	3/10 (30%)	4/10 (40%)	Fused Hybrid Win
Science: Experimental Method	2/10 (20%)	5/10 (50%)	5/10 (50%)	Fused Hybrid Co-Win
Science: Ecological Systems	3/10 (30%)	3/10 (30%)	4/10 (40%)	Fused Hybrid Win

3. Situational Commonsense: 15 HellaSwag Benchmarks (150 Questions)

Benchmark Name	Llama-3.2-1B-Instruct	SmolLM2-1.7B-Instruct	SmolLlama-1.7B-Fused	Result / Winner
Commonsense: Athletics Sports	4/10 (40%)	4/10 (40%)	4/10 (40%)	3-Way Tie
Commonsense: Home Culinary	3/10 (30%)	3/10 (30%)	3/10 (30%)	3-Way Tie
Commonsense: Diy Crafts	4/10 (40%)	4/10 (40%)	5/10 (50%)	Fused Hybrid Win
Commonsense: Social Interaction	2/10 (20%)	5/10 (50%)	6/10 (60%)	Fused Hybrid Win
Commonsense: Automotive Transport	3/10 (30%)	5/10 (50%)	5/10 (50%)	Fused Hybrid Co-Win
Commonsense: Performing Arts	1/10 (10%)	6/10 (60%)	4/10 (40%)	SmolLM2 Win
Commonsense: Outdoor Recreation	2/10 (20%)	4/10 (40%)	3/10 (30%)	SmolLM2 Win
Commonsense: Workplace Office	2/10 (20%)	4/10 (40%)	4/10 (40%)	Fused Hybrid Co-Win
Commonsense: Childcare Education	1/10 (10%)	3/10 (30%)	4/10 (40%)	Fused Hybrid Win
Commonsense: Pet Animal Care	3/10 (30%)	5/10 (50%)	5/10 (50%)	Fused Hybrid Co-Win
Commonsense: Technology Electronics	3/10 (30%)	5/10 (50%)	6/10 (60%)	Fused Hybrid Win
Commonsense: Medical Health	2/10 (20%)	6/10 (60%)	5/10 (50%)	SmolLM2 Win
Commonsense: Gardening Nature	3/10 (30%)	5/10 (50%)	3/10 (30%)	SmolLM2 Win
Commonsense: Travel Navigation	2/10 (20%)	4/10 (40%)	4/10 (40%)	Fused Hybrid Co-Win
Commonsense: Household Chores	4/10 (40%)	2/10 (20%)	3/10 (30%)	Llama-3.2 Win

4. Quantitative Logic: 10 GSM8K Benchmarks (100 Questions)

Benchmark Name	Llama-3.2-1B-Instruct	SmolLM2-1.7B-Instruct	SmolLlama-1.7B-Fused	Result / Winner
Math: Single Step Arithmetic	4/10 (40%)	3/10 (30%)	4/10 (40%)	Fused Hybrid Co-Win
Math: Two Step Word Problems	2/10 (20%)	2/10 (20%)	2/10 (20%)	3-Way Tie
Math: Multi Operator Algebra	2/10 (20%)	4/10 (40%)	2/10 (20%)	SmolLM2 Win
Math: Rates And Speed	1/10 (10%)	0/10 (0%)	2/10 (20%)	Fused Hybrid Win
Math: Percentage And Discount	2/10 (20%)	3/10 (30%)	2/10 (20%)	SmolLM2 Win
Math: Fractional Reasoning	0/10 (0%)	4/10 (40%)	2/10 (20%)	SmolLM2 Win
Math: Geometry And Area	1/10 (10%)	2/10 (20%)	2/10 (20%)	Fused Hybrid Co-Win
Math: Financial Budgeting	2/10 (20%)	1/10 (10%)	0/10 (0%)	Llama-3.2 Win
Math: Combinatorics Counting	3/10 (30%)	1/10 (10%)	2/10 (20%)	Llama-3.2 Win
Math: Multi Tier Optimization	0/10 (0%)	4/10 (40%)	2/10 (20%)	SmolLM2 Win

5. World Knowledge: 10 Semantic Benchmarks (100 Questions)

Benchmark Name	Llama-3.2-1B-Instruct	SmolLM2-1.7B-Instruct	SmolLlama-1.7B-Fused	Result / Winner
Semantic: Causal Chains	2/10 (20%)	8/10 (80%)	7/10 (70%)	SmolLM2 Win
Semantic: Spatio Temporal Order	3/10 (30%)	8/10 (80%)	7/10 (70%)	SmolLM2 Win
Semantic: Material Properties	5/10 (50%)	7/10 (70%)	7/10 (70%)	Fused Hybrid Co-Win
Semantic: Biological Instincts	3/10 (30%)	5/10 (50%)	7/10 (70%)	Fused Hybrid Win
Semantic: Linguistic Pragmatics	4/10 (40%)	5/10 (50%)	4/10 (40%)	SmolLM2 Win
Semantic: Economic Consequences	4/10 (40%)	7/10 (70%)	7/10 (70%)	Fused Hybrid Co-Win
Semantic: Physical Plausibility	6/10 (60%)	7/10 (70%)	7/10 (70%)	Fused Hybrid Co-Win
Semantic: Social Conventions	4/10 (40%)	3/10 (30%)	3/10 (30%)	Llama-3.2 Win
Semantic: Tool Affordances	6/10 (60%)	2/10 (20%)	5/10 (50%)	Llama-3.2 Win
Semantic: Environmental Adaptations	2/10 (20%)	4/10 (40%)	4/10 (40%)	Fused Hybrid Co-Win

Test Suite 2: Strict Zero-Shot Blind HellaSwag Benchmark (20 Unseen Items)

Evaluated under frozen Pareto-optimal $\alpha^* = 0.020$ on 20 unseen items from Rowan/hellaswag:

Evaluation Metric	Parent 1: Llama-3.2-1B	Parent 2: SmolLM2-1.7B	SmolLlama-1.7B-Fused	Outcome
Accuracy	3 / 20 (15.0%)	8 / 20 (40.0%)	8 / 20 (40.0%)	+25.0% over Llama-3.2, 100% SmolLM2 Retention

Test Suite 3: Multi-Anchor Hub Benchmark Sweep (15 Questions)

Benchmark Dataset	Llama-3.2-1B-Instruct	Single-Splice Hybrid	SmolLM2-1.7B-Instruct	Multi-Anchor Hybrid
ARC-Challenge (`ai2_arc`)	1 / 5 (20.0%)	2 / 5 (40.0%)	4 / 5 (80.0%)	3 / 5 (60.0%)
GSM8K (`gsm8k`)	0 / 5 (0.0%)	0 / 5 (0.0%)	1 / 5 (20.0%)	1 / 5 (20.0%) (under $\alpha=0.025$)
MMLU (`mmlu`)	0 / 5 (0.0%)	1 / 5 (20.0%)	2 / 5 (40.0%)	1 / 5 (20.0%)
Total Test Suite	1 / 15 (6.7%)	3 / 15 (20.0%)	7 / 15 (46.7%)	4 / 15 (26.7%)

Test Suite 4: Official Hugging Face Leaderboard Ceiling References

Benchmark	Standard Metric	Llama-3.2-1B-Instruct	SmolLM2-1.7B-Instruct
ARC-Challenge	Accuracy (0-shot / Average)	59.4%	60.5%
GSM8K	Exact Match (5-8 shot CoT)	44.4%	48.2%
MMLU	Macro Average (5-shot)	49.3%	52.1%
HellaSwag	Accuracy (0-shot / 10-shot)	41.2%	68.7%

Quickstart & Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "YOUR_USERNAME/SmolLlama-1.7B-Fused"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "Explain the fundamental theorem of calculus in simple terms."
messages = [
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.6, top_p=0.9)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Model Details

Base Architecture: LlamaForCausalLM (SmolLM2-1.7B layout)
Parameters: 1.71 Billion
Precision: bfloat16
Context Length: 8,192 tokens
VRAM Requirements: ~3.4 GB VRAM for inference (runs comfortably on 4GB–6GB consumer GPUs)

Downloads last month: 196

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for taniota/Warlock-1.7B

HuggingFaceTB/SmolLM2-1.7B-Instruct

unsloth/Llama-3.2-1B-Instruct

Merge model

this model

Quantizations

2 models