LFM2.5-230M-Uncensored

A 230M parameter language model with refusal direction steering applied. This model is configured to minimize refusal behaviors through layer-wise steering techniques.

Model Overview

• Parameters: 230M
• Architecture: LFM2 hybrid (8 convolutional + 6 attention layers)
• Context length: 128,000 tokens
• Vocabulary size: 65,536
• Training data: 19T tokens
• Knowledge cutoff: Mid-2024
• Languages: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish

Steering Configuration

This model has been steered using orthogonalization at layer 9 to adjust refusal behavior patterns in the embedding space.

Technical Details

Configuration	Value
Model Type	lfm2
Number of parameters	230M
Number of layers	14
Hidden size	1024
Number of attention heads	16
Context length	128,000 tokens
Vocabulary size	65,536

Chat Template

The model uses a ChatML-compatible format:

<|startoftext|><|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Your message here<|im_end|>
<|im_start|>assistant

Use tokenizer.apply_chat_template() to format messages automatically.

Quick Start

Basic Inference

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Suchinthana/LFM2.5-230M-Uncensored"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    device_map="auto",
)
model.eval()

# Format messages using chat template
messages = [
    {"role": "user", "content": "What is your name?"}
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

# Generate outputs
with torch.no_grad():
    output_ids = model.generate(
        input_ids=input_ids,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.95,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

# Decode and print response
response = tokenizer.decode(
    output_ids[0][input_ids.shape[-1]:],
    skip_special_tokens=True
).strip()
print(response)

Chat Interface

For an interactive chat interface:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

class ChatEngine:
    def __init__(self, model_id: str):
        self.tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
        self.model = AutoModelForCausalLM.from_pretrained(
            model_id,
            trust_remote_code=True,
            device_map="auto",
        )
        self.model.eval()
        
        if self.tokenizer.pad_token_id is None:
            self.tokenizer.pad_token_id = self.tokenizer.eos_token_id

    def generate(self, message: str, history=None):
        # Build message history
        messages = history or []
        messages.append({"role": "user", "content": message})
        
        # Tokenize and generate
        input_ids = self.tokenizer.apply_chat_template(
            messages,
            add_generation_prompt=True,
            return_tensors="pt"
        ).to(self.model.device)
        
        with torch.no_grad():
            output_ids = self.model.generate(
                input_ids=input_ids,
                max_new_tokens=512,
                temperature=0.7,
                top_p=0.95,
                pad_token_id=self.tokenizer.pad_token_id,
                eos_token_id=self.tokenizer.eos_token_id,
            )
        
        new_tokens = output_ids[0][input_ids.shape[-1]:]
        return self.tokenizer.decode(new_tokens, skip_special_tokens=True).strip()

# Usage
engine = ChatEngine("Suchinthana/LFM2.5-230M-Uncensored")
response = engine.generate("Hello!")
print(response)

Tool Use

Function calling is supported through the chat template:

messages = [
    {
        "role": "system",
        "content": 'Available tools: [{"name": "calculator", "description": "Performs math", "parameters": {}}]'
    },
    {"role": "user", "content": "What is 2+2?"}
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():
    output = model.generate(input_ids, max_new_tokens=512)

response = tokenizer.decode(output[0], skip_special_tokens=True)
print(response)

Requirements

torch>=2.0.0
transformers>=5.0.0

Install with:

pip install torch transformers

Model Specifications

Steering Method

• Method: Orthogonalization
• Applied at Layer: 9
• Effect: Reduces refusal behaviors in model responses

Performance Characteristics

This is a lightweight 230M parameter model designed for edge deployment. It supports:

• Long context lengths (up to 128k tokens)
• Multi-language responses
• Efficient inference on CPU and GPU