Siyam025/MODEL-2

crypto-ts-transformer

Overview

crypto-ts-transformer is a Transformer-based time-series regressor trained to predict short-term cryptocurrency price movement (close returns) using OHLCV features plus a market sentiment signal. It is designed for research and prototyping in quant strategies and short-horizon forecasting.

Model Architecture

• Encoder-Decoder Transformer adapted for regression.
• Input: sliding window of past seq_length timesteps (default 96).
• Output: horizon-length continuous predictions (default 12 timesteps ahead).
• Key hyperparameters:
  • d_model: 128
  • encoder layers: 4
  • decoder layers: 2
  • attention heads: 8

Intended Use

• Short-term forecasting of normalized returns for research, backtesting, and feature engineering.
• Use for generating features (alpha signals) in algorithmic trading pipelines.
• Not intended as direct trading advice. Always validate with a simulation/backtest and risk-control logic.

Limitations

• Trained on historical data; past performance does not imply future results.
• Model ignores transaction costs, slippage, and execution constraints.
• Vulnerable to regime shifts (e.g., sudden market news, exchange outages).
• Requires rescaling of inputs consistent with training (scaler: standard). Mismatched scaling will degrade outputs.

Example Code

import torch
import numpy as np

# Load model (load state_dict into your TransformerRegressor class)
# This repository contains only the config and weights; loading code is expected to be part of your infra.
from your_package import TransformerRegressor  # placeholder

config = {
    "input_size": 6,
    "d_model": 128,
    "num_encoder_layers": 4,
    "num_decoder_layers": 2,
    "num_attention_heads": 8,
    "dim_feedforward": 512,
    "dropout": 0.1,
    "output_size": 1,
    "seq_length": 96,
    "prediction_horizon": 12
}

model = TransformerRegressor(**config)
state = torch.load("pytorch_model.bin", map_location="cpu")
model.load_state_dict(state)
model.eval()

# Example input: (batch, seq_length, input_size)
dummy = np.random.randn(1, config["seq_length"], config["input_size"]).astype("float32")
x = torch.from_numpy(dummy)
with torch.no_grad():
    preds = model(x)  # shape: (batch, horizon, output_size)
print(preds.shape)