You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Qwen-3.5-2B Chess Fine-tuned (SFT)

Overview

This is a fine-tuned version of Qwen-3.5-2B on chess move prediction using Supervised Fine-Tuning (SFT). The model was trained on the Thytu/ChessInstruct dataset.

Model Details

  • Base Model: Qwen/Qwen3.5-2B
  • Fine-tuning Method: Supervised Fine-Tuning (SFT)
  • Precision: bfloat16

Training Configuration

  • Learning Rate: 1e-5
  • Batch Size: 4 (per device)
  • Gradient Accumulation Steps: 1
  • Epochs: 3
  • Optimizer: AdamW with cosine learning rate scheduler
  • Warmup Steps: 100
  • Weight Decay: 0.01

Training Dynamics

Train Loss Curve Eval Loss Curve

Dataset

The model was trained on Thytu/ChessInstruct, a chess instruction dataset containing:

  • Chess positions with move annotations
  • Game sequences with context
  • Various chess task categories (move prediction, advantage evaluation, etc.)

Task Categories

Category Description
FIND_NEXT_BEST_MOVE Predict the next best move
FIND_ADVANTAGED_PLAYER Identify who has advantage
FIND_FINAL_SCORE Predict game result
MLM_ON_MOVES Fill in missing moves
FIND_LAST_MOVE Find the final move
SORT_FENS Sort positions by game order

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "GL3MON/Qwen3.5-2B-chess-finetuned"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Example: Predict chess moves
messages = [
    {"role": "system", "content": "You are a chess assistant. Provide the best move in SAN format."},
    {"role": "user", "content": "What is the best move for white in the position: rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1?"}
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(input_ids, max_new_tokens=50, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Evaluation

The model was evaluated on a validation split of the ChessInstruct dataset. The training achieved good convergence with the loss decreasing steadily over 3 epochs.

License

MIT License

Citation

@software{ChessLM2026,
  title = {ChessLM - Qwen-3.5-2B Chess Assistant},
  author = {GL3MON},
  year = {2026},
  url = {https://huggingface.co/GL3MON/Qwen3.5-2B-chess-finetuned}
}

References

Downloads last month
-
Safetensors
Model size
2B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for GL3MON/Qwen3.5-2B-chess-finetuned

Finetuned
Qwen/Qwen3.5-2B
Finetuned
(308)
this model

Collection including GL3MON/Qwen3.5-2B-chess-finetuned